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EXECUTIVE  SUMMARY 


The  employment  of  unmanned  aerial  vehicles  (UAVs)  by  the  Israeli  Defense 
Forces  during  the  Peace  for  Galilee  campaign  in  1982,  and  by  United  States  forces  during 
Operations  Desert  Shield  and  Desert  Storm  in  1990-91,  provides  a  proof  of  concept  that 
relatively  low  cost  UAVs  can  provide  effective  surveillance,  reconnaissance,  target 
acquisition  and  fire  support  adjustment  missions.  The  United  States  has  employed  UAVs 
subsequently  in  operations  in  Somalia,  Haiti,  Bosnia,  and  most  recently  in  Kosovo.  The 
Department  of  the  Navy’s  doctrine  of  Operational  Maneuver  from  the  Sea  will  place 
greater  reliance  on  aerial  battlefield  surveillance  from  "over  the  horizon"  or  expeditionary 
sites  ashore  in  order  to  control  a  greater  operational  area  with  less  force  concentration. 
However,  the  current  Naval  UAV  platform,  the  Pioneer,  is  beset  by  an  unacceptable 
mishap  rate.  Since  its  fielding  in  operational  units  in  1986,  the  Pioneer  Class  A  flight 
mishap  rate  is  385  mishaps  per  100,000  flight  hours.  This  stands  in  stark  contrast  to  that 
of  manned  Naval  Aviation  where  the  rate  is  approximately  two  Class  A  flight  mishaps 
per  100,000  flight  hours. 

Schmidt  &  Parker  (1995),  while  working  at  the  Naval  Safety  Center  in  Norfolk, 
Virginia,  have  identified  that  human  factors  related  issues  cause  half  of  the  Naval  UAV 
mishaps.  Seagle  (1997)  applies  the  Human  Factors  Analysis  and  Classification  System 
(HFACS)  taxonomy  to  UAV  mishap  reports  in  order  to  improve  human  factors  mishap 
investigation,  reporting  and  analysis.  The  HFACS  taxonomy,  based  upon  the  Reason 
(1990)  "Swiss  Cheese"  model  of  accident  causation,  captures  the  latent  conditions  that 
"set  the  stage"  for  active  failures  that  lead  to  mishaps. 
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This  study  refines  the  coding  of  UAV  mishaps  in  accordance  with  the  HFACS 
taxonomy.  Once  the  mishaps  are  parsed,  the  factors  influencing  the  mishap  rate  and  their 
resultant  costs  are  computed.  These  results  are  used  to  conduct  a  stochastic  model 
simulation  of  annual  UAV  flight  operations  in  order  to  isolate  those  categories  of  mishaps 
that  contribute  most  to  increased  budgetary  costs  and  decreased  mission  readiness.  The 
study  places  UAV  mishap  reports  from  FY86  to  FY98  into  categories  using  the  HFACS 
taxonomy.  Particular  emphasis  is  placed  on  the  FY93  to  FY98  period  when  Naval  UAVs 
first  came  under  the  cognizance  of  the  Naval  Aviation  Safety  Program.  Then,  mishap 
rates  and  probabilities  are  estimated  for  the  various  categories.  Thereupon,  these 
probability  distributions  are  used  as  the  input  to  a  stochastic  model  for  the  simulation  of 
annual  flight  operations.  The  model  output  is  the  annual  mishap  costs  associated  with  a 
particular  category,  and  a  resulting  mission  readiness  index. 

Intervention  strategies  are  assigned  to  associated  category  of  mishap  causation. 
Comparing  the  annual  mishap  cost,  readiness  index,  and  the  feasibility  associated  with  a 
particular  intervention  strategy,  Fleet  users  and  program  managers  can  determine  what 
intervention  strategies  are  most  appropriate.  Some  strategies  are  specific  to  the  Pioneer 
system,  for  example,  an  engine  remanufacturing  or  upgrading,  or  an  electronic 
weatheiproofing  modification.  Other  strategies,  such  as  improved  aircrew  coordination 
training  and  personnel  assignment  stabilization,  transcend  the  Pioneer  system,  and  are 
applicable  to  all  follow-on  UAV  configurations. 

The  mishap  categorization  phase  of  this  study,  data  coding  and  classification,  was 
conducted  independently  of  previous  studies  (Schmidt  &  Parker,  1995;  Seagle,  1997). 
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The  relative  frequency  of  mishap  causes  agrees  with  the  previous  classifications,  and 
validates  their  findings.  Even  though  mishap  reports  remain  vague,  and  lack  granularity 
at  identifying  lower,  root  causes,  there  is  general  agreement  in  the  classification  of  the 
mishap  in  each  independent  analysis. 

The  second  phase  of  the  study,  stochastic  model  simulation  of  annual  flight 
operations,  finds  that  electro-mechanical  causes  have  a  low  impact  on  mission  readiness 
although  they  account  for  approximately  one-quarter  of  UAV  mishaps.  On  the  other 
hand,  unsafe  acts,  unsafe  conditions  for  flight  operations,  and  unsafe  supervision  have  a 
more  significant  impact  on  equipment  repair  and  replacement  costs,  and  mission 
readiness.  The  study  concludes  that  addressing  the  human  factors  related  issues  through 
increased  aircrew  coordination  training,  the  use  of  simulators  for  mission  rehearsal, 
personnel  stabilization,  and  the  development  of  a  UAV  career  path,  will  have  a  greater 
impact  on  controlling  cost  and  improving  readiness.  The  study  also  generates  the 
budgetary  costs  of  selected  mishap  categories.  These  costs  can  be  compared  to  the  cost  of 
implementing  a  particular  intervention  strategy.  Thereupon,  the  program  manager  can 
chose  the  most  appropriate  strategies  for  implementation.  While  in  isolation,  no  single 
intervention  strategy  will  eliminate  mishap  occurrences;  their  implementation  will 
increase  the  density  of  the  "safety  net"  surrounding  UAV  operations. 
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I.  INTRODUCTION 


A.  OVERVIEW 

From  the  advent  of  modem  military  operations  through  the  present  day,  successful 
commanders  have  relied  on  knowledge  of  the  battlefield  and  intelligence  of  the  enemy’s 
disposition  in  order  to  employ  their  force  to  achieve  victory  (Marine  Aviation  Weapons 
and  Tactics  Squadron  One  [MAWTS-1],  1997).  During  the  19th  century,  in  the  American 
Civil  War  and  the  Franco-Prussian  War,  hot  air  balloons  were  employed  in  order  to  get  a 
bird’s  eye  view  of  the  battlefield.  They  added  a  third  dimension  to  the  intelligence 
collecting  effort.  Early  in  the  20th  century,  the  invention  of  the  airplane  replaced  the 
restrictive  and  vulnerable  use  of  balloons.  During  the  First  World  War,  manned  aircraft 
were  employed  as  airborne  intelligence  collectors.  Later,  in  the  1960’s  in  response  to 
increasingly  lethal  air  defense  networks  and  enormously  developing  technology, 
reconnaissance  satellites  have  taken  over  the  role  of  aerial  surveillance  at  the  strategic  and 
operational  levels  of  warfare.  Presently,  Unmanned  Aerial  Vehicles  (UAVs)  are  used  to 
augment  satellite  systems  by  providing  near  real-time,  tactical  aerial  reconnaissance. 
However,  the  current  Naval  UAV,  the  Pioneer,  is  beset  by  an  unacceptable  mishap  rate. 
This  thesis  addresses  several  potential  intervention  strategies  designed  to  mitigate  these 
mishaps,  lower  their  budgetary  cost  and  improve  operational  readiness. 

B.  AERIAL  RECONNAISSANCE 

The  U-2  "spy  plane"  was  developed  during  the  early  1950s  in  order  to  monitor 
Soviet  ICBM  development  and  deployment  (Jones,  1997).  The  U.S.  conducted  U-2  over 
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flights  of  the  Soviet  Union  since  1955.  On  2  May  1960,  Francis  Gary  Powers  was  shot 
down  in  a  U-2  over  the  Soviet  Union  by  an  SA-2  "Guideline"  missile.  The  shoot  down  of 
Powers,  and  the  failed  attempt  at  covering  up  the  mission,  proved  to  be  a  devastating 
blow  to  the  U.S.’s  international  prestige.  A  second  U-2  was  shot  down  over  Cuba  during 
the  Missile  Crisis  on  27  October  1962,  while  attempting  to  determine  the  status  of  Soviet 
nuclear  missiles.  Consequently,  the  nation  became  increasingly  wary  of  manned 
reconnaissance.  However,  the  only  reconnaissance  systems  in  development  were  the  SR- 
71  Blackbird  and  the  CORONA  spy-satellite.  Both  of  these  systems  were  designed  for 
strategic  level  reconnaissance  only. 

The  U.S.  made  an  initial  serious  attempt  at  an  airborne  tactical  reconnaissance 
UAV  during  the  Vietnam  War  (Jones,  1997).  The  U.S.  Air  Force  experimented  with 
launching  the  Teledyne-Ryan  developed  Lightning  Bug  UAV  from  MC-130's  in  order  to 
conduct  aerial  reconnaissance.  By  the  end  of  the  war,  the  Lightning  Bug  UAV  had  grown 
into  a  vehicle  for  providing  real-time  video,  electronic  intelligence  (ELJNT)  collection, 
electronic  countermeasures  (ECM),  communications  intelligence  (COMINT)  and 
psychological  operations  (PSYOPS)  leaflet  dropping.  However  interest  in  UAVs  waned 
as  the  war  wound  down,  and  the  Department  of  Defense  (DoD)  trimmed  budgets  and 
force  structure. 

Nevertheless,  U.S.  lessons  learned  in  Vietnam  did  not  go  unheeded.  During  that 
same  period,  Israel  Aircraft  Industries  (IAI)  began  the  successful  development  UAVs  for 
the  Israeli  Defense  Forces  (IDF)  (Kumar,  1997).  These  Israeli  developed  and 
manufactured  UAVs  were  first  employed  in  combat  in  1982  during  the  Peace  for  Galilee 
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campaign  in  Lebanon.  During  an  attack  against  Syrian  forces  in  the  Bekaa  Valley,  the 
Israeli  Air  Force  (IAF)  launched  decoy  missiles  to  stimulate  the  Syrian  air  defense 
system.  As  the  Syrians  responded  to  this  perceived  attack,  Israeli  Mastiff  and  Scout 
UAVs  with  electro-optical  and  radar-detecting  payloads  were  able  to  locate  and  target 
Syrian  missile  sites.  Once  target  locations  were  confirmed,  and  during  the  Syrian  missile 
reload  cycle,  the  IAF  launched  A-4,  F-4  and  KFIR  aircraft  to  attack  these  targets.  The 
Israelis  destroyed  19  SAM  batteries  and  86  MiG  aircraft  while  only  losing  one  aircraft  of 
their  own,  effectively  dismembering  the  entire  Syrian  air  defense  network. 

The  early  1980s  saw  a  resurgence  in  enthusiasm  for  UAVs  within  DoD  as  a  result 
of  the  Israeli’s  successful  UAV  employment  in  Lebanon.  U.S.  peacekeeping  operations  in 
Beirut  (1982-3),  Operation  Urgent  Fury  (Grenada,  1983),  Operation  Eldorado  Canyon 
(Libya,  1986)  all  highlighted  a  requirement  for  an  inexpensive,  over  the  horizon, 
unmanned  reconnaissance  capability  for  the  on-scene  tactical  commander.  In  July  1985, 
the  Secretary  of  the  Navy,  John  Lehman  directed  the  expeditious  acquisition  of  UAVs  for 
fleet  operations.  By  December  1985,  the  U.S.  Navy  procured  the  Pioneer  UAV  system 
developed  by  IAI.  The  Pioneer  UAV  is  the  next  generation  UAV  following  the  Mastiff 
that  was  employed  in  the  Peace  for  Galilee  campaign.  The  U.S.  Navy  and  Marine  Corps 
began  establishing  UAV  units  in  1986.  The  first  Pioneer  overseas  deployment  occurred 

in  December  1986,  aboard  the  U.S.S.  Iowa  (Pioneer  UAV  Inc.,  1999). 

/ 

The  first  time  the  Pioneer  UAV  saw  combat  action  was  in  Operations  Desert 
Shield  and  Desert  Storm  (Melson,  Englander  &  Dawson,  1992).  During  that  conflict,  six 
UAV  units  were  deployed  to  the  Kuwaiti  Theater  of  Operations  (KTO)  -  two  Navy,  three 
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Marine  Corps  and  one  Army.  Together,  the  units  flew  336  missions,  accruing  985  flight 
hours.  Only  three  UAVs  were  hit  by  enemy  anti-aircraft  artillery  (AAA),  resulting  in  the 
loss  of  only  one  UAV  from  enemy  action.  General  Walter  Boomer,  USMC  (ret.)  who 
commanded  the  Marine  Expeditionary  Force  (MEF)  in  the  Kuwaiti  theater  summed  up 
the  effect  of  UAV  employment  during  the  war  when  he  said,  "The  Pioneer  UAV  was  the 
most  significant  intelligence  collection  source  within  I  MEF." 

Pioneer  UAV  units  were  later  deployed  for  Operations  Restore  and  Continue 
Hope  in  Somalia,  Operation  Uphold  Democracy  in  Haiti,  and  Operation  Joint  Endeavor 
in  Bosnia.  Their  successful  mission  performance  reinforced  the  potential  of  exploiting 
this  technology  to  support  future  combat  operations  and  operations  other  than  war 
(OOTW)  (Jenkins,  1998).  The  Department  of  the  Navy’s  doctrine  of  "Operational 
Maneuver  from  the  Sea"  (OMFTS)  will  place  greater  reliance  on  battlefield  surveillance 
from  "over  the  horizon"  and  expeditionary  sites  ashore  in  order  to  control  a  greater 
operational  area  with  less  force  concentration.  Relatively  inexpensive  reconnaissance 
UAVs  with  the  ability  to  conduct  missions  in  hostile  airspace,  without  putting  pilots  into 
harm’s  way  is  a  key  element  of  OMFTS  (Marine  Corps  Combat  Development  Command 
[MCCDC],  1999). 

C.  THE  PIONEER  UAV  SYSTEM 

A  Pioneer  UAV  system  consists  of  five  Pioneer  air  vehicles,  a  ground  control 

station  (GCS),  a  portable  control  station  (PCS),  a  tracking  communications  unit  (TCU),  a 
data  link,  two  remote  receiver  stations  (RRS)  and  a  reconnaissance  payload.  The  system 
can  be  operated  aboard  specially  configured  U.S.S.  Austin  Class  Landing  Platform  Dock 
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(LPD-4)  ships  or  from  prepared  airstrips  ashore.  While  UAVs  do  not  carry  any  ordnance, 
they  can  perform  as  forward  observers  of  indirect  fire  support  assets  -  offensive  attack 
aviation,  artillery  and  naval  surface  fire  support.  Figure  1  is  a  representation  of  how 
UAVs  can  be  deployed  during  an  amphibious  operation. 


Figure  1:  Pioneer  UAV  Concept  of  Operations 


Within  DoD,  only  the  U.S.  Navy,  Marine  Corps  and  the  Defense  UAV  Training 
Command  (DUTC)  operate  the  Pioneer  UAV  (MAWTS-1,  1997).  The  Navy's  UAV  unit 
is  VC-6,  headquartered  in  Norfolk,  Virginia;  however,  the  UAV  elements  are  located  at 
Webster  Field,  Patuxet  River,  Maryland.  From  there,  they  deploy  as  detachments  with  a 
complete  Pioneer  system  aboard  ship.  There  are  two  Marine  Corps  UAV  units.  VMU-1 
is  located  in  Twenty-Nine  Palms,  California,  and  VMU-2  is  headquartered  at  MCAS 
Cherry  Point,  North  Carolina.  The  training  command,  DUTC,  is  located  in  Fort 
Huachuca,  Arizona.  The  U.S.  Army  deactivated  their  Pioneer  UAV  units  in  anticipation 
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of  procuring  the  Hunter  (IAI)  or  Outrider  (Alliant  Techsystems)  UAV  system.  However, 
both  programs  were  cancelled  because  they  failed  to  meet  desired  size,  weight  and 
corrosion  control  specifications  (Sherman,  1998). 

Pioneer  UAV  units  are  currently  tasked  to  provide  the  following  missions: 

a)  Reconnaissance,  surveillance  and  target  acquisition  (RSTA). 

b)  Adjusting  indirect  fire  (Artillery,  Naval  Surface  Fire  Support). 

c)  Collect  Battle  Damage  Assessment  (BDA). 

d)  Support  security  operations  (e.g.,  convoy  escort,  monitor  enemy 

avenues  of  approach  and  named  areas  of  interest  (NAIsj. 

The  term  "unmanned"  is  a  misnomer  when  applied  to  the  Unmanned  Aerial 
Vehicle  system  because  UAV  operations  involve  many  remote  participants.  The  essential 
members  of  a  UAV  crew  include  a  Mission  Commander,  an  Internal  Pilot,  a  Payload 
Operator  and  an  External  Pilot.  Additionally  launch  and  recovery  teams  and  maintenance 
personnel  will  be  involved  in  flight  operations  (Joint  UAV  Training  Operations 
Procedures  &  Standardization  [JUAVTOPS],  1997).  A  detailed  description  of  the 
Pioneer  UAV  system,  capabilities  and  crew  responsibilities  is  attached  as  Appendix  A. 

D.  PIONEER  UAV  SAFETY  RECORD 

Schmidt  &  Parker  (1995),  of  the  Naval  Safety  Center  (NSC),  begin  the  initial 
effort  at  improving  UAV  operational  readiness  by  focusing  on  mishap  prevention.  Their 
study  examines  the  107  mishaps  that  occurred  between  1986  and  1993.  The  breakdown 
of  causal  factors  is  illustrated  in  Figure  2.  This  research  indicates  a  significant  number 
(approximately  59%)  of  mishaps  occur  as  a  result  of  electromechanical  problems. 
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Human  error  accounts  for  nearly  one-third  of  these  mishaps.  Their  research  indicates  the 
following  factors  present  significant  safety  concerns:  crew  selection  and  training; 
aeromedical  readiness;  pilot  proficiency/currency;  personnel  shortages;  operational  tempo 
(OPTEMPO);  human  error  in  teamwork  and  aircraft  control. 


Figure  2:  FY  86-  FY  94  Naval  Pioneer  UAV  Mishap  Causal  Factors  (Schmidt  &  Parker,  1995) 
Schmidt  &  Parker  (1995)  recommend  the  following  corrective  actions  be  taken  to 
decrease  the  mishap  rate:  aeromedical  screening  and  monitoring;  criteria  based  selection 
process;  UAV  crew  coordination  training;  take  off,  landing  and  external  pilot  (EP) 
simulators;  inclusion  into  OPNAVINST  3710.7P  NATOPS  oversight  and  improved 
human  systems  integration.  These  recommendations  have  been  implemented  with 
varying  degrees  of  success.  Aeromedical  screening  is  conducted  prior  to  being  assigned 
to  the  UAV  community.  In  addition,  aptitude  testing  is  conducted  to  assess  the  flight 
potential  of  incoming  aircrew.  Aircrew  coordination  training  (ACT)  is  currently  in 
progress  and  constantly  evolving.  A  take  off  and  landing  drone  was  developed  to  enable 
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EPs  to  fly  an  air  vehicle  without  having  the  entire  system  activated.  In  FY  94,  UAV 
flight  operations  were  included  in  the  Naval  Aviation  Safety  Program  (OPNAVTNST 
3750.6Q). 

These  improvements  have  marginally  decreased  the  mishap  rate.  However, 
improvement  is  still  required.  Initially,  aeromedical  screening  takes  place  but  a  flight 
surgeon  is  not  assigned  to  UAV  units  to  continue  aeromedical  education  and  crew 
monitoring.  The  lack  of  operational  experience  and  personnel  stability  in  UAV  units 
limits  ACT  because  lessons  learned  are  difficult  to  capture  and  implement.  Additionally, 
emergency  action  simulation  drills  are  not  possible  because  there  is  no  crew  simulator. 
Air  vehicle  drones,  called  MiGs  by  their  crews,  do  not  have  the  same  aerodynamic 
characteristics  of  the  Pioneer  air  vehicle.  Thus,  while  the  MiG  training  is  effective  in 
training  the  EPs  in  basic  flight  controls  and  procedures,  the  discrepancies  between  its 
response  and  that  of  the  real  Pioneer  vehicle  may  reinforce  improper  handling  and  result 
in  a  negative  learning  experience. 

Seagle  (1997)  continues  the  work  of  Schmidt  &  Parker.  He  applies  the  Human 
Factors  Mishap  Classification  System  (HFACS)  to  analyze  UAV  mishaps  from  1986  to 
1997.  His  research  studies  the  203  mishap  investigation  reports  from  1986  through  1997, 
determining  that  88  include  human  related  causal  factors.  His  categorization  of  mishaps 
is  illustrated  in  Table  1.  His  work  provides  insight  into  the  cause  of  human  factors 
related  mishaps  and  clarifies  the  broad,  generalized  "human  error"  labels.  Seagle  goes 
further  by  demonstrating  that  although  the  primary  cause  of  an  accident  may  have  been 
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electro-mechanical  in  nature,  a  latent  cause  was  due  potentially  to  human  factors,  either 


contributing  to  the  mishap,  or  failing  to  correct  a  condition  that  led  to  the  accident.  • 


CAUSAL  FACTOR 

CODE 

# 

FREQ 

Unsafe  Act 

■■ 

52 

59.1% 

Intended 

UAI 

6 

6.8% 

Mistake 

UAIM 

34 

38.6% 

Violation 

6 

i  6.8% 

Unintended 

UAU 

46 

52.3% 

Slip 

UAUS 

2 

2.2% 

14 

15.9% 

Unsafe  Condition 

40 

45,5% 

Aeromedical 

3Z3SHH 

18 

20.4% 

CRM 

UCC 

24 

27.2% 

Readiness 

UCV 

6 

6.8% 

Unsafe  Supervision 

54 

61.4 

Unforeseen 

usu 

30 

34.1% 

Foreseen 

USF 

41 

46.5% 

Human  Factors 

HI 

88 

Table  1 :  FY86  -  FY97  UAV  Mishaps  Parsed  by  Causal  Category  (Seagle,  1 997) 

Seagle  confirms  Schmidt  &  Parker’s  recommendations  for  a  full  crew  simulator  to 
enhance  the  rehearsal  of  ACT  and  flight  emergency  drills.  He  also  reports  a  trend  in  the 
following  categories:  loss  of  situational  awareness,  lack  or  loss  of  depth  perception, 
visual  illusions,  self-medication  and  fatigue.  In  1996,  the  Navy’s  Bureau  of  Medicine 
(BUMED)  incorporated  medical  standards  for  UAV  aircrew  to  address  these  factors. 
Seagle  also  proposes  an  automatic,  "hands  off'  landing  system  for  runway  arrestment  and 
embarked  net  recoveries  with  an  override  capability  for  the  EP  or  IP  in  the  case  of 
degraded  operations.  He  attributes  the  lack  of  urgency  in  implementation  of  these  efforts 
to  the  UAV  community  having  no  "champion"  at  the  flag  officer  level.  The  relatively 
inexperienced  and  young  officers  who  serve  in  the  UAV  community  have  not  reached  (or 
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may  never  reach)  the  rank  where  they  can  affect  change,  unlike  officers  in  manned 
aviation  careers. 

E.  RESEARCH  OBJECTIVE 

The  objective  of  this  study  is  to  further  identify  the  causal  factors  resulting  in  the 
unacceptable  number  of  UAV  mishaps.  Particular  emphasis  is  placed  on  human  factors 
related  mishaps.  Once  identified  the  study  constructs  a  stochastic  model  to  evaluate 
mishap  intervention  initiatives  with  the  goal  of  mishap  reduction  in  terms  of  cost  and 
mission  readiness.  The  results  of  the  model  are  presented  to  allow  decision-makers  to 
focus  on  specific  accident  causation  categories  and  to  choose  the  most  efficient  and 
effective  intervention  strategies  for  further  development. 

F.  STATEMENT  OF  THE  PROBLEM 

Since  its  fielding  in  1986,  the  Pioneer  has  accumulated  a  Class  A  mishap  rate  of 

385  mishaps  per  100,000  flight  hours.  When  this  is  compared  to  the  manned  aviation 
Class  A  mishap  rate  of  approximately  two  mishaps  per  100,000  flight  hours,  one  sees  the 
Homeric  proportions  of  this  unacceptable  situation.  The  excessive  UAV  mishap  rate 
translates  into  significant  budgetary  cost,  degradation  in  mission  readiness,  and  a 
perception  of  unreliability  by  fleet  users  and  those  whom  they  support.  Steps  must  be 
taken  to  bridge  the  gap  between  conceptual  capability  and  actual  performance.  The 
bottom  line  is  to  achieve  a  dramatic  reduction  in  the  UAV  mishap  rate  to  an  acceptable 
level,  to  sustain  mission  readiness,  and  to  minimize  mishap  costs.  Specific  research  goals 
include  the  following: 
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1.  The  classification  of  UAV  mishaps  under  the  current  aviation  mishap 
taxonomy.  Additionally,  the  identification  the  human  factors  characteristics  that 
significantly  impact  the  UAV  mishap  rate. 

2.  The  development  of  a  stochastic  model  of  UAV  mishaps  which  can  be  used  to 
accurately  represent  mishap  occurrences. 

3.  The  use  of  the  model  to  analyze  the  effects  of  mishap  reduction  intervention 
strategies. 

4.  The  identification  of  the  impact  of  a  particular  intervention  strategy  on  cost 
savings  and  mission  readiness  improvement. 

G.  DEFINITIONS 

Mishap.  A  naval  mishap  is  an  unplanned  event  or  series  of  events  directly 
involving  naval  aircraft,  which  results  in  $10,000  or  greater  cumulative  damage  to  naval 
aircraft  or  personnel  injury.  Aviation  flight  mishaps  are  divided  into  one  of  three 
categories  based  on  the  severity  of  the  damage,  and  the  cost  incurred  by  the  mishap.  The 
definitions  governing  each  class  of  mishap  is  given  in  Naval  Publication  OPNAVINST 
3750.6Q. 

Class  A  Mishap:  A  class  A  mishap  occurs  when  the  total  amount  of  damage 
exceeds  $1,000,000  or  if  the  air  vehicle  is  destroyed.  Because  the  total  cost  of  a  Pioneer 
UAV  is  approximately  $1.1  million,  a  class  A  mishap  will  only  occur  if  the  UAV  is 
damaged  beyond  repair  or  lost.  Class  A  mishaps  include  being  lost  at  sea,  or  destruction 
of  the  air  vehicle  if  it  forcefully  impacts  terrain.  Loss  of  a  UAV  during  a  combat  mission 
is  not  classified  as  a  class  A  mishap,  but  rather,  a  combat  loss. 
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Class  B  Mishap:  This  category  is  used  when  the  total  cost  of  damage  is  at  least 
$200,000  but  less  than  $1,000,000.  Usually,  a  UAV  class  B  mishap  occurs  when  there  is 
serious  damage  to  the  air  vehicle,  or  if  the  payload  is  damaged.  The  cost  of  the 
surveillance  camera  alone  ranges  from  $300,000  to  $800,000  depending  on  whether  it  is 
the  day  EO  or  night  FLIR  camera.  The  variance  of  the  cost  depends  upon  whether  the 
payload  is  repairable  or  permanently  damaged. 

Class  C  Mishap:  This  category  is  used  when  the  total  cost  of  damage  is  at  least 
$10,000  but  less  than  $200,000.  For  UAV  mishaps,  this  situation  occurs  when  there  is  a 
small  amount  of  damage  to  the  air  vehicle  possibly  from  a  hard  landing,  or  striking 
another  object.  Usually,  swapping  out  or  remanufacturing  parts  can  repair  these  types  of 
mishaps,  keeping  costs  low. 

H.  SCOPE  AND  LIMITATIONS 

The  scope  of  this  research  is  limited  to  the  Defense  UAV  Training  Command,  and 
U.S.  Navy  and  Marine  Corps  fleet  Pioneer  UAV  squadrons.  However,  the  results  of  this 
analysis  will  be  pertinent  to  all  UAV  operations  of  current  and  future  systems  that 
integrate  a  human  component  to  conduct  the  mission.  Detailed  mishap  analysis  is 
conducted  from  FY93  through  FY98.  In  FY93,  UAVs  were  incorporated  into  the  Naval 
Aviation  Safety  system,  and  mishap  reporting  procedures  were  standardized.  Prior  to 
FY93,  UAV  mishap  reports  contained  limited  descriptive  information,  rendering  detailed 
analysis  nearly  impossible.  Mishap  causes  were  typically  described  as  pilot  error, 
electrical  or  mechanical  failure  with  little  amplifying  information. 
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The  contents  of  the  thesis  are  presented  in  the  following  order:  Chapter  Two  is  a 
literature  review  of  three  human  error  causation  models.  They  serve  to  provide 
background  to  the  HFACS  model  used  by  the  NSC,  which  in  turn  is  discussed  in  detail 
with  specific  examples  of  UAV  applications.  Chapter  Two  also  discusses  accident 
investigations  and  analysis,  mishap  intervention  strategies,  and  stochastic  modeling. 
Chapter  Three  describes  the  methods  used  to  estimate  statistical  parameters,  which  are 
inputs  to  the  stochastic  model.  Further,  the  simulation  methodology  is  developed. 
Chapter  Four  includes  the  mishap  database  construction,  parameter  estimates  and  the 
output  of  the  stochastic  model  simulation.  Conclusions  and  recommended  courses  of 
action  are  presented  in  Chapter  Five. 
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II.  LITERATURE  REVIEW 

A.  ACCIDENT  CAUSATION  THEORIES 

The  Asia-Pacific  Safety  Magazine  (March,  1995),  published  by  the  Australian 
Bureau  of  Air  Safety  Investigations,  reports  "between  70%  and  80%  of  [mishap] 
occurrences  contain  a  human  factors  element."  The  article  continues,  "Indeed  it  can  be 
argued  that  human  factors  are  involved  in  all  occurrences."  In  a  system  designed,  built, 
maintained  and  operated  by  humans,  only  a  fraction  of  one  percent  of  the  mishaps  can  be 
attributed  to  factors  beyond  human  control  (Bruggink,  1996). 

Because  of  society's  obvious  goal  to  reduce  the  number  of  aviation  mishaps  and 
eventually  their  overall  prevention,  much  research  has  been  dedicated  to  determining 
accident  causation,  and  developing  a  safe  environment  where  the  probability  of  having  an 
accident  is  minimized  (Harle,  1993).  Nevertheless,  the  results  of  aviation  accident 
investigations  often  limit  conclusions  to  phrases  such  as  "pilot  error,"  "failure  to  see  and 
avoid,"  "improper  use  of  controls,"  or  "failure  to  observe  and  adhere  to  established 
standard  operating  procedures  (SOPs). 

"However,  effective  accident  intervention  and  prevention  requires  more  than 
identifying  who  is  culpable.  In  order  to  implement  a  safety  conscious  program,  a  better 
understanding  of  the  context  in  which  these  individuals  faced  accident  conducive 
circumstances  is  required. 

Within  the  last  thirty  years,  three  accident  causation  theories  stand  out  for  their 
merits  in  dissecting  the  myriad  of  contributing  causal  factors  that  create  the  context  of  an 
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incident.  These  three  theories  are  Edward’s  (1977)  "SHEL"  model;  Helmreich’s  (1990) 
model;  and  the  Reason’s  (1990)  "Swiss  Cheese"  model.  Each  of  the  three  theories 
attempts  to  explore  the  latent  or  removed  factors  that  influence  the  mishap.  The  main 
focus  is  on  the  "chain  of  events"  and  surrounding  circumstances  that  lead  to  an  accident. 

1.  The  SHEL  Model 

The  "SHEL"  (Software,  Hardware,  Environment  and  Liveware)  model  is  first 
introduced  by  Edwards  (1972)  and  later  modified  by  Hawkins  (1984).  The  SHEL  model 
places  emphasis  on  the  human  being  and  its  interfaces  with  the  other  components  of  the 
man-machine-environment  system.  Each  component  of  the  SHEL  model  represents  the 
components  of  a  modem  technological  system  as  depicted  in  Figure  3. 


Figure  3:  The  SHEL  Model 


The  human  operator  is  set  at  the  center  of  the  model,  and  must  interact  with  each 
of  the  four  external  components.  The  edges  of  the  blocks  are  not  simple  or  straight, 
indicating  that  the  human  must  be  matched  with  each  component  in  order  to  function 
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properly.  A  mismatch  between  blocks,  or  an  improper  fit  is  a  potential  cause  for  human 
error.  In  order  to  analyze  the  human  factors  aspects  of  an  environment,  one  must  look  at 
both  the  individual  component  blocks,  and  also  their  interface  (Harle,  1993).  The 
following  is  a  brief  description  of  each  component  of  the  model: 

a)  Human  Operator 

The  human  operator  is  the  hub  of  the  SHEL  model.  As  such,  analysis  of 
the  individual  must  incorporate  four  categories:  physical,  physiological,  psychological, 
and  psychosocial.  Physically,  one  must  determine  if  the  individual  is  capable  of 
performing  the  required  task,  and  if  there  are  any  impediments  or  limitations  to  successful 
task  performance.  Physiologically,  an  individual  must  be  prepared  to  conduct  the  task. 
This  category  includes  the  items  of  proper  nutrition,  alcohol  or  drug  use,  tobacco  use, 
stress  and  fatigue,  and  the  effect  these  have  on  an  individual  or  crew’s  ability  to  perform 
and  make  appropriate  decisions.  Psychologically,  an  individual  must  be  capable  of 
mentally  executing  the  task.  Knowledge  of  what  is  required,  and  the  confidence  to 
perform  the  task  must  be  established.  Moreover,  the  workload  must  be  appropriate  to  an 
individual’s  information  processing  and  attention  capabilities.  Finally,  psychosocial 
factors  impact  human  performance.  Stress,  pressure  from  a  supervisor,  the  workplace 
climate  and  personal  issues  all  influence  one’s  reaction  in  a  potentially  dangerous 
situation. 

b)  Liveware 

The  liveware  interface  is  the  operator’s  relationship  to  the  other  individuals 

in  the  workplace.  In  aviation,  this  is  often  referred  to  as  crew  resource  management 
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(CRM)  and  is  addressed  later.  Liveware  also  relates  to  teamwork,  morale  and  the  overall 
command  climate.  Obviously,  healthy  interpersonal  relationships  among  a  crew,  or 
between  individuals  and  their  supervisors  are  essential  for  a  safe  and  effective  work 
environment. 

c)  Hardware 

The  hardware  interface  addresses  the  human  machine  interface  (HMI). 
The  HMI  includes  workspace  configuration,  displays,  controls,  seat  design  and 
configuration,  visibility  and  climatic  conditions.  The  physical  work  environment  impacts 
crew  orientation,  information  processing,  cognition  and  execution  of  the  task.  Similarities 
of  component  design  and  their  physical  layout  can  affect  effective  scan  patterns,  and 
facilitate  correlation  of  input  data.  The  hardware  interface  is  the  focus  of  ergonomic  and 
anthropometrical  study. 

d)  Software 

The  software  interface  is  the  relationship  between  the  individual  and  all 
supporting  systems  found  in  the  workplace.  This  category  includes  not  only  computer 
software  design,  but  also  regulations,  manuals,  checklists,  and  SOPs.  These  items  must 
be  user  friendly  and  understandable  to  the  human  operator.  Automation  is  also  a 
significant  contributor  to  the  software  human  interface.  An  entirely  automated  system 
may  have  the  tendency  to  breed  complacency  and  boredom  for  an  operator  resulting  in 
decreased  vigilance.  On  the  other  hand,  the  lack  of  automation  can  cause  task  saturation 
for  an  operator  in  extreme  situations,  also  resulting  in  an  unsafe  environment. 
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e)  Environment 

The  environmental  interface  is  the  relationship  between  the  operator  and 
the  internal  and  external  environment.  The  internal  environment  includes  temperature, 
lighting,  noise,  vibration  and  air  quality.  The  external  environment  includes  visibility, 
weather  and  terrain.  For  military  applications,  the  external  environment  includes  the 
combat  situation  during  which  operations  must  be  executed.  The  surrounding 
environment  has  significant  impact  on  individual  and  crew  motivation,  attention, 
judgment  and  performance. 

2.  The  Helmreich  Model 

Helmreich  advised  the  International  Civil  Aviation  Organization  (ICAO) 
Commission  of  Inquiry  on  the  human  factors  aspects  of  an  Air  Ontario  flight  accident  that 
took  place  in  Dryden,  Ontario  in  1990  (Zotov,  1996).  The  advantage  of  his  theory  is  that 
it  eliminates  an  opportunity  for  regulators  and  organizations  to  argue  that  their  actions 
should  not  be  discussed  in  the  accident  report.  It  is  frequently  argued  that  an  accident  is 
the  end  result  of  a  chain  of  events:  if  a  causal  factor  can  be  removed  and  the  accident 
could  still  have  occurred,  then  ipso  facto  that  factor  can  not  be  causal.  The  Helmreich 
model  illustrates  that  an  accident  is  the  accumulation  of  factors,  rather  than  their  chaining 
together,  which  affects  a  crew’s  performance. 

The  Helmreich  model  can  be  envisioned  as  a  series  of  concentric  circles 
surrounding  an  operator  or  crew.  Each  ring  potentially  influences  the  crew  and  may 
cause  a  degradation  of  performance.  The  four  levels  of  influence  are  the  regulatory 
environment,  the  organizational  environment,  the  physical  environment,  and  the  crew 
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environment  (see  figure  4).  Accidents  occur  as  the  result  actions  taken  within  the  context 
of  this  crew  environment. 


Figure  4:  Helmreich  Model  of  Accident  Causation 

a)  Regulatory  Environment 

The  regulatory  environment  is  the  guidance  for  conducting  operations. 
Within  Naval  aviation,  this  includes  NATOPS  procedures,  the  Training  and  Readiness 
(T&R)  manual,  unit  SOPs,  operations  orders  and  commanders  guidance.  These 
regulations  guide  supervisors  on  how  to  conduct  mission  planning  and  execution  within 
certain  limitations.  Examples  of  these  regulations  are  proficiency  and  currency 
requirements,  crew  rest  and  approved  flight  profiles. 

b)  Organizational  Environment 

The  organizational  environment  includes  crew  composition  and  its 

performance.  Consistent  training,  personnel  stability,  OPTEMPO  and  leadership 

contribute  to  the  organizational  environment.  On  the  micro  level,  the  interface  between 
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operations  and  maintenance  personnel,  and  on  the  macro  level  between  a  unit  and  is 
higher  and  adjacent  units  within  the  command  and  control  structure  greatly  influence  the 
organizational  environment  and  the  niche  into  which  a  crew  fits. 

c)  Physical  Environment 

The  physical  environment  is  consistent  with  the  internal  and  external 
environments  of  the  SHEL  model.  It  also  includes  the  physical  condition  of  equipment  at 
the  time  of  an  incident,  and  what  affect  that  had  on  mission  performance.  The  model 
recognizes  that  the  surroundings  of  an  individual  or  crew  have  a  significant  impact  on  the 
vigilance  and  responsiveness  of  the  operator. 

d)  The  Crew  Environment 

The  crew  environment  comprises  the  interpersonal  coordination  and 
communication  within  a  crew.  The  model  also  extends  the  crew  definition  to  include  its 
interfaces  with  external  control,  such  as  air  traffic  (ATC)  and  enroute  controllers. 
Anyone  associated  with  the  conduct  of  a  mission  is  de  facto  part  of  the  extended  crew. 
The  crew  environment  is  also  influenced  by  CRM.  Finally,  the  crew  is  further  broken 
down  into  individual  components.  Individuals  possess  their  own  performance  strengths, 
weaknesses  and  vulnerabilities. 

The  Helmreich  model  of  accident  causation  is  crew  centered.  However,  instead  of 
"blaming"  the  crew  or  individuals  on  the  crew,  it  looks  at  the  context  of  the  accident  that 
arises  from  the  faulty  actions  or  decisions  by  external  agencies.  These  contributing 
factors,  although  potentially  missing  from  the  "chain  of  events"  are  equally  influential  in 
an  accident  occurrence  (Zotov,  1996). 
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3.  The  "Swiss  Cheese" Model 

The  "Swiss  Cheese"  model  is  the  outgrowth  of  research  examining  human  error 
by  Reason  (1990).  The  model  discusses  the  layers  of  defense  used  to  protect  against 
accident  occurrences.  Reason’s  model  identifies  four  layers  that  potentially  contribute  to 
an  accident:  organizational  influences,  unsafe  supervision,  unsafe  conditions  and 
individuals  performing  unsafe  acts  (see  Figure  5). 


Figure  5:  Reason’s  Swiss  Cheese  Model 

In  an  ideal  world,  the  defensive  layers  would  be  intact,  preventing  the  "accident 
trajectory,"  as  depicted  in  the  diagram  by  the  arrow,  from  passing  through  to  the  accident 
event.  However,  each  layer  has  weaknesses  and  gaps  that  are  revealed  by  the  holes.  In 
the  real  world,  these  holes  are  not  fixed  and  static,  otherwise  they  could  be  identified  and 
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repaired.  Reason  (1998)  describes  the  holes  as  dynamic  and  in  a  constant  state  of  flux. 
Local  conditions  drive  which  defensive  layer  comes  in  and  out  of  the  frame  at  a  particular 
time. 

Reason  (1998)  attributes  the  holes  as  either  active  failures,  or  latent  conditions. 
Active  failures  are  either  violations  or  errors  that  occur  in  the  immediate  vicinity  of  the 
accident  occurrence.  They  are  performed  by  the  operators  -  pilots,  air  traffic  controllers, 
police  officers,  control  room  operators,  maintenance  personnel,  and  so  on.  The  discovery 
and  mitigation  of  this  active  failure  immediately  prior  to  the  accident  would  most  likely 
prevent  the  accident  from  happening.  Twenty  years  ago,  the  discovery  of  this  unsafe  act 
would  have  ended  an  accident  investigation.  However  in  today's  climate,  unsafe  acts  are 
seen  more  as  consequences  than  principle  causes.  It  is  recognized  that  people  working  in 
complex  systems  make  errors  or  violate  procedures  for  reasons  that  go  beyond  an 
individual.  These  causes  are  called  latent  conditions. 

Latent  conditions  can  be  present  for  many  years  before  they  combine  with  local 
circumstances  and  active  failures  to  penetrate  they  systems  layers  of  defense.  Examples 
of  latent  conditions  include  poor  design,  improper  training  and  supervision,  undetected 
manufacturing  defects  or  poor  design,  unworkable  procedures,  or  improper  automation. 
At  the  macro  level,  government,  regulator  or  corporate  policy  shapes  the  organizational 
culture,  creating  the  error  producing  factors  within  an  individual  environment. 

Latent  conditions  are  present  in  all  systems  and  are  an  inevitable  part  of  the 
organizational  culture.  Latent  conditions  are  not  bad  policy  decisions,  but  can  result  from 
the  demands  of  a  limited  budget  or  manpower  management  constraints.  These  latent 


23 


conditions  can  lie  dormant  for  years  and  have  no  impact  until  they  become  manifested  at 
a  time  where  particular  weaknesses  in  the  defense  become  exposed.  In  contrast  to  active 
failures,  which  tend  to  be  unique  to  a  specific  event,  latent  conditions  can  go 
unrecognized  and  can  contribute  to  a  number  of  different  accidents.  Latent  conditions 
increase  the  likelihood  of  active  failures  through  the  creation  of  local  factors  allowing 
error  and  violations  to  occur  (Reason,  1998). 

B.  HUMAN  FACTORS  ANALYSIS  AND  CLASSIFICATION  SYSTEM 

The  Human  Factors  Analysis  and  Classification  System  (HFACS)  taxonomy  as 
developed  by  Shappell  &  Wiegmann  (1997)  at  the  Naval  Safety  Center  (NSC),  is  based 
on  Reason’s  concept  of  human  error  in  accident  causation.  HFACS  also  incorporates, 
albeit  to  a  lesser  extent,  components  of  the  SHEL  and  Helmreich  model  to  define  the 
context  surrounding  an  accident.  The  HFACS  taxonomy  will  be  added  to  the  next 
publication  of  OPNAVINST  3750  and  become  the  standardized  methodology  adopted  by 
the  Naval  Aviation  Safety  Program  for  human  factors  mishap  investigation.  HFACS 
attempts  to  capture  the  context  in  which  an  accident  occurs  by  categorized  failures  into 
four  separate  tiers  depicted  in  Figure  6.  These  tiers  are  organizational  influences,  unsafe 
supervision,  unsafe  conditions,  and  unsafe  acts.  For  the  purposes  of  this  discussion,  each 
tier  is  viewed  through  the  context  of  naval  UAV  operations. 
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Unsafe  Acts 


Figure  6:  HFACS  Mishap  Causation  Categories 


1.  Organizational  Influences 

The  first  HFACS  tier  of  mishap  causation  is  organizational  influences. 
Organizational  influences  are  often  difficult  to  quantify  and  can  rarely  be  tied  as  a 
specific  cause  to  an  accident.  However,  their  existence  certainly  contributes  to  the 
circumstances  surrounding  an  accident.  The  author  discussed  these  influences  with  UAV 
crewmembers  at  VMU-1,  VMU-2  and  VC-6.  Among  aviators,  there  is  a  perception  that  a 
UAV  tour  shows  a  lack  of  competitiveness  with  one’s  peers  who  are  in  a  flying  billet  in 
their  primary  airframe.  Being  assigned  to  the  UAV  unit  relegates  them  to  the  status  of  a 
second-class  citizen.  In  contrast,  several  see  the  UAV  field  as  an  opportunity  to  excel  in  a 
challenging  and  developing  community.  As  such,  they  channel  their  drive  and  energy  to 
guarantee  success. 

Another  organizational  influence  is  the  lack  of  a  UAV  specialty  and  career  path. 
Until  1996,  UAV  enlisted  personnel  did  not  have  a  specific  rating  or  military  operational 
specialty  (MOS).  As  a  result,  personnel  were  assigned  to  a  UAV  unit  with  no  previous 
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background,  underwent  training  and  left  the  unit  three  years  later,  never  to  return  again. 
Initial  accessions  to  UAV  units  were  recruited  by  advertising  for  individuals  whose  hobby 
was  remote  control  airplanes.  Initially,  a  unit  could  be  made  up  of  truck  drivers, 
mechanics,  yeomen,  or  any  combination  of  backgrounds.  The  establishment  of  enlisted 
specialties  has  improved  the  situation.  Nevertheless,  there  are  no  officer  specialties  or 
career  paths.  It  is  still  the  case  that  an  officer  will  only  serve  one  tour  in  a  UAV  unit 
before  leaving  the  community.  Typically,  all  officers  in  a  UAV  unit,  including  the 
commanding  officer,  will  be  new  to  the  community  upon  assignment  (MAWTS-1, 1997). 

A  third  organizational  influence  within  the  UAV  community  is  the  perception  by 
others  in  the  aviation  field.  Besides  the  normal  rivalries  that  exist  among  communities, 
aviators  typically  relegate  UAV  crews  to  being  the  lowest  on  the  totem  pole.  Often,  their 
inclusion  in  aviation  planning  and  operations  in  regarded  as  a  nuisance  by  those  who  do 
not  understand  the  unique  integration  requirements  that  UAV  operations  must  address. 
These  organizational  conditions,  unhealthy  at  times,  can  create  an  environment 
susceptible  to  unsafe  operations  and  potentially  lead  to  mishap  causation. 

2.  Unsafe  Supervision 

Unsafe  supervision  is  the  second  HFACS  tier  that  contributes  to  an  actual  mishap 
occurrence.  Unsafe  supervision  can  be  both  unforeseen  and  foreseen.  Unforeseen  unsafe 
supervision  includes  unrecognized  unsafe  operations,  inadequate  documentation  and 
inadequate  design.  Unrecognized  unsafe  operations  result  from  a  supervisor  not 
recognizing  an  unsafe  act  or  condition.  For  example,  a  supervisor  may  not  be  aware  of 
accumulated  fatigue  among  aircrew  or  maintenance  personnel  in  a  unit.  Instead  of  taking 
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corrective  action,  the  situation  is  ignored.  Also,  by  not  being  aware  of  influences  on 
individuals,  crew  assignments  and  personnel  management  suffer.  An  example  is  the 
supervisor  who  is  not  aware  that  a  crewmember  has  a  sick  spouse  or  child,  a  recent  death 
in  the  family,  or  marital  difficulty.  As  a  result,  the  supervisor  does  not  know  to  take  these 
adverse  mental  conditions  into  account  when  making  crew  assignments. 

Inadequate  documentation  refers  to  unknown  "bugs"  in  the  system.  Designed  for 
combat,  the  Pioneer  was  not  fully  tested  until  Operation  Desert  Storm.  Operational 
testing  and  training  are  designed  to  closely  resemble  combat,  but  cannot  replicate  it. 
During  Operation  Desert  Storm,  the  Marine  Pioneer  units  operated  as  part  of  a  full  scale 
Marine  Expeditionary  Force  (MEF).  A  microwave  antenna/transmitter  was  located  at  the 
UAV  airstrip  for  communications.  The  microwave  signal  caused  electro-mechanical 
interference  (EMI)  with  the  UAV  uplink  and  downlink  causing  loss  of  vehicle  control 
resulting  in  crashes  on  several  occasions.  Because  the  Pioneer  had  not  previously 
operated  in  close  proximity  to  this  specific  communications  equipment,  this  condition 
was  neither  realized  nor  documented. 

Inadequate  design  is  an  extension  of  inadequate  documentation  in  that  an 
inadequate  condition  exists  that  is  unintentional.  The  engineers  who  designed  the 
equipment  may  not  have  anticipated  requirements  for  certain  performance  characteristics 
or  capabilities.  Although  it  may  seem  inconceivable  for  a  naval  UAV,  the  Pioneer  was 
not  built  to  fly  through  rain  or  visible  moisture.  It  was  designed  by  the  Israeli  Aircraft 
Industries  (IAI)  for  operations  in  arid  areas  around  Israel.  It  has  a  laminated  wooden 
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propeller  that  delaminates  in  moisture,  and  the  electronic  components  on  the  air  vehicle 
are  not  waterproof.  This  inadequate  design  has  been  the  cause  of  several  UAV  mishaps. 

Foreseen  unsafe  supervision  is  the  mismanagement  of  individuals  at  the  personal 
level.  It  includes  the  lack  of  or  inadequate  supervision,  the  failure  to  correct  a  known 
problem,  or  a  supervisory  violation.  Foreseen  unsafe  supervision  is  a  lack  of  leadership 
and  guidance  in  a  crew  or  in  the  entire  unit  that  creates  an  unsafe  situation  for  flight 
operations.  Within  a  unit,  this  can  be  an  underlying  condition  that  is  reflected  in  lack  of 
discipline,  operational  focus  or  morale.  Unsafe  supervision  is  rarely  an  isolated  instance, 
but  is  symptomatic  of  underlying  conditions,  which  can  all  lead  to  a  mishap. 

3.  Unsafe  Conditions 

Reason  (1990)  addresses  the  category  of  unsafe  conditions  in  accident  causation, 
however  Shappell  &  Wiegmann’s  (1997)  HFACS  taxonomy  further  subdivides  these 
conditions  into  aeromedical,  crew  resource  management  and  readiness  violations.  The 
following  paragraphs  characterize  each  of  these  subdivisions.  Additionally,  each 
subdivision  is  discussed  within  the  context  of  UAV  community  issues. 

a)  Aeromedical  Conditions 

Aeromedical  conditions  include  the  physiological  and  mental  condition  of 
individuals,  and  their  physical  and  mental  limitations.  The  physiological  condition  of  an 
individual  includes  the  functioning  of  their  sensory  system  and  physical  condition.  A 
UAV  crewmember  may  experience  spatial  disorientation  or  visual  illusions  caused  by  the 
remote  controls  of  the  air  vehicle.  Unlike  manned  aviation  where  pilots  incorporate 
vestibular  inputs  with  visual  cues  to  perceive  the  attitude  and  motion  of  the  aircraft,  a 
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UAV  operator  does  not  have  those  inputs.  In  order  to  operate  effectively  as  a 
crewmember,  there  are  required  sensory  thresholds  that  must  be  maintained.  In  addition 
to  the  senses,  other  physical  conditions  can  affect  the  crewmember.  Though  not 
exhaustive,  medications,  fatigue,  change  in  circadian  rhythm,  hypoglycemia,  use  of 
alcohol,  or  vitamin  deficiencies  can  cause  a  person  to  be  physically  unqualified  to 
conduct  a  mission  (Edwards,  1990). 

The  adverse  mental  condition  of  a  crewmember  is  also  an  aeromedical 
condition  that  can  adversely  affect  mission  performance.  The  effects  of  stress  and  mental 
workload  on  human  performance  are  widely  documented.  Both  can  cause  a  loss  of 
situational  awareness,  mental  fatigue  and  task  saturation.  Additionally,  personality  traits 
and  attitudes  such  as  overconfidence,  complacency,  misplaced  motivation,  or  a  desire  to 
please,  cause  an  adverse  mental  state.  Increased  OPTEMPO,  and  associated  family 
separations,  financial  concerns,  operational  or  combat  fatigue  and  competition  among 
members  of  a  crew,  all  potentially  combine  to  create  the  adverse  mental  conditions  that 
can  cause  an  accident  (Hawkins,  1987).  The  final  aeromedical  contribution  to  unsafe 
conditions  is  the  physical  and  mental  limitations  of  the  crewmember.  Individuals  must  be 
physically  and  mentally  screened  for  selection  as  UAV  aircrew  in  order  to  prevent  the 
environment  for  the  previously  discussed  conditions  to  occur. 

b)  Crew  Resource  Management 

Jensen  (1995)  defines  crew  resource  management  (CRM)  as  the  effective 
use  of  all  resources  (hardware,  software,  and  liveware)  to  achieve  safe  and  efficient  flight 
operations.  The  resources  that  a  crew  manages  are  people  (other  crewmembers). 
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equipment  (instruments  and  controls)  and  other  items  such  as  charts,  checklists  and 
operational  manuals.  UAV  operations,  as  stated  earlier,  involve  a  crew  of  at  least  four, 
and  normally  more  crewmembers.  The  competency  and  experience  of  these  individuals, 
their  supervision,  their  interpersonal  communication  and  the  remote  aspect  of  the  air 
vehicle  from  the  operational  control  site  combine  to  make  unique  CRM  demands  on  a 
UAV  crew.  The  crew  must  communicate  via  an  intercom  system  (ICS)  and 
crewmembers  often  cannot  establish  eye  contact  with  one  another.  Additionally,  not  all 
crewmembers  have  access  to  flight  instrument  information.  As  the  number  of 
crewmembers  goes  up,  the  potential  for  conflicting  interpretations  of  information  also 
increases,  potentially  causing  confusion  and  indecisive  actions. 

c)  Readiness 

Readiness  violations  refer  to  violations  of  standard  operating  procedures 
(SOPs),  rules,  and  instructions  designed  to  provide  a  safe  operating  environment  for  flight 
operations.  Among  other  things,  NATOPS  defines  regulations  on  crew  rest,  self- 
medication  and  alcohol  consumption.  In  the  Marine  Corps,  the  Training  and  Readiness 
(T&R)  Manuals  (MCO  3500.21:  Volumes  I  and  VI),  define  crew  proficiency  and 
currency  requirements  for  an  individual  conducting  a  specified  mission.  Violations  of 
any  of  these  regulations  can  create  unsafe  conditions  for  a  mission. 

4.  Unsafe  Acts 

The  fourth  HFACS  tier  is  unsafe  acts,  which  can  be  classified  as  either  intended 
or  unintended.  Intended  acts  are  either  mistakes  or  violations,  whereas,  unintended  acts 
are  either  slips  or  lapses.  The  following  paragraphs  characterize  each  of  these 
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subdivisions.  Additionally,  each  subdivision  is  discussed  within  the  context  of  UAV 
community  issues. 

a)  Intended  Acts 

Mistakes  are  failures  to  formulate  the  correct  intentions,  and  can  result 
from  shortcomings  of  perception,  memory  and  cognition.  In  these  cases,  the  intended 
action  is  wrong.  Knowledge  based  mistakes  are  caused  by  failures  to  understand  the 
situation  and  arriving  at  an  incorrect  course  of  action.  In  this  situation  a  UAV  operator 
may  be  saturated  with  raw  information,  and  may  not  be  able  to  process  the  data  to 
formulate  the  correct  actions.  Also,  because  of  inexperience,  an  operator  does  not  have 
the  capacity  to  assimilate  the  information  given  to  formulate  a  correct  response. 

In  contrast,  someone  with  more  experience  who  misapplies  a  rule  under 
certain  conditions  usually  makes  a  rule-based  error.  A  rule-based  decision  can  be  likened 
to  if... then  logic.  These  mistakes  normally  occur  in  one  of  three  ways.  The  first  is  that  a 
rule  is  followed,  but  exceptions  to  the  situation  are  not  noted.  An  example  is  a  UAV 
operator  who  has  been  operating  from  a  shore  location  and  is  now  operating  at  sea. 
Emergency  procedures  that  are  learned  and  reinforced  through  training  ashore  may  not  be 
applicable  or  desirable  at  sea.  A  mistake  occurs  when  the  operator  applies  a  land-based 
procedure  at  sea  and  an  accident  occurs.  The  second  type  of  rule  based  mistake  is  when 
the  if...  part  of  the  situation  is  misinterpreted,  and  the  ...then  action  is  inappropriately 
applied.  And  finally,  the  third  type  of  rule  based  mistake  is  when  the  if...  observation  is 
correct  and  the  ...then  action  is  incorrectly  applied.  (Wickens,  1992) 
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In  contrast  to  mistakes,  violations  are  the  willful  breaking  of  rules  or 
procedures.  Violations  can  be  categorized  into  routine  or  exceptional  violations.  A 
routine  violation  tends  to  be  habitual  in  nature  and  is  typical  of  an  individual’s  behavioral 
repertoire.  An  example  of  a  UAV  related  routine  violation  is  when  a  crewmember 
habitually  fails  to  follow  the  unit  SOP  or  NATOPS  procedures.  This  can  be  as  harmless 
as  failing  to  brief  a  certain  emergency  procedure  during  a  pre-flight  brief.  Lack  of 
supervision  and  correction  action  acerbates  the  violation  by  passively  reinforcing 
incorrect  procedures.  On  the  other  hand,  an  exceptional  violation  is  an  isolated  departure 
from  authority,  neither  typical  of  the  individual  nor  condoned  by  supervisors.  The 
deliberate  decision  to  ignore  directions  from  an  air  controller  is  an  example  of  an 
exceptional  violation. 

b)  Unintended  Actions 

A  slip  is  an  unintended  error  in  which  the  correct  intention  is  incorrectly 
carried  out,  as  opposed  to  a  mistake  where  the  incorrect  intention  is  correctly  carried  out. 
For  example,  the  internal  pilot  (IP)  uses  dials  to  control  the  speed  and  altitude  of  the  air 
vehicle.  Intending  to  adjust  the  UAV  airspeed  and  inadvertently  changing  the  altitude 

I 

dial  is  an  example  of  a  slip.  According  to  Wickens  (1992),  slips  occur  for  three  reasons: 
(1)  the  intended  action  involves  a  slight  departure  from  the  routine,  frequently  performed 
action;  (2)  some  characteristics  of  the  stimulus  environment  or  action  sequence  closely 
relate  to  the  inappropriate,  but  more  frequent  action;  and  (3)  the  action  sequence  is 
automated  and  therefore,  not  monitored  closely  by  attention. 
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Finally,  a  lapse  is  the  failure  to  carry  out  an  action.  Memory  failure, 
memory  overload,  or  interruption  can  cause  a  lapse.  Prior  to  every  UAV  mission,  all 
members  of  the  UAV  crew  perform  various  tasks  as  outlined  in  a  preflight  checklist.  If 
that  sequence  of  events  is  interrupted  unexpectantly,  requiring  the  crew  or  an  individual 
to  divert  attention  and  then  come  back  to  the  pre-flight  sequence,  a  particular  step  may  be 
skipped.  This  momentary  distraction  can  cause  a  lapse  to  occur  with  unknown 
consequences. 

C.  ACCIDENT  INVESTIGATIONS,  ANALYSIS,  AND  REPORTS 

Bruggink’s  (1996)  research  in  civil  aviation  points  out  that  accident  reporting  is 
too  preoccupied  with  reactive,  formal  responses  to  stated  accident  causes.  Typical 
investigations  focus  on  the  black  and  white  elements  of  accident  causes.  Since  the  role  of 
human  factors  is  often  a  gray  area,  it  can  seldom  be  accommodated  by  the  rules  of 
evidence  favored  by  investigating  authorities.  Emphasis  is  placed  upon  direct  cause- 
effect  relationships,  and  causal  statements  become  official  designators  of  blame.  The 
inclusion  of  contributing  factors  to  causal  statements  perpetuates  a  distinction  between 
primary  causes  and  contributing  factors  that  has  the  effect  of  lessening  their  relationship 
to  the  accident.  McAdams,  a  senior  and  respected  member  of  the  National  Transportation 
and  Safety  Board  (NTSB)  commented  after  a  1978  mid-air  collision  investigation  that:  "A 
contributing  factor  is  not  a  primary  cause;  it  is  more  remote  and  does  not  carry  the  same 
weight  or  implications  as  that  of  a  probable  cause."  (Bruggink,  1996)  As  a  result,  the 
context  of  the  situation  is  dismissed  easier  than  the  individual  acts  themselves. 
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Zotov  (1996)  also  concludes  that  there  exists  reluctance  to  closely  analysis  human 
factors  related  causal  factors.  Often  investigations  lead  to  personal  attacks,  legal 
obstruction  and  corporate  pressure  on  the  investigating  authority  if  the  entire  "system"  is 
under  investigation.  A  corporation  will  attempt  a  legal  response  that  frees  them  from 
culpability  by  using  the  "chain  of  causation"  approach  to  accident  occurrences. 
Additionally,  the  legal  concept  of  "remoteness  of  damage"  can  make  it  difficult  to  present 
the  concept  of  a  complex  network  of  interacting  events  that  caused  an  accident. 

Mayer  &  Ellingstad  (1992)  state  that  accident  databases  frequently  describe 
attributes  of  the  physical  environment  and  equipment,  but  that  detailed  analysis  of 
accident  causes,  including  human  factors  information,  are  frequently  not  represented 
because  they  are  too  difficult  to  obtain  and  code.  Engineers  and  front-line  operators 
design  most  accident  reporting  systems  with  limited  backgrounds  in  human  factors.  This 
results  in  a  lack  of  suitability  for  addressing  human  factors  issues.  Naval  Aviation  is 
continuing  to  attempt  addressing  this  shortcoming  through  the  Naval  Aviation  Safety 
Program. 

The  purpose  of  the  Naval  Aviation  safety  program  is  to  preserve  both  human  and 
material  resources  in  order  to  enhance  operational  readiness.  In  order  to  accomplish  this 
goal,  damage  and  injury  must  be  addressed  to  mitigate  the  hazards  inherent  in  flight 
operations  (OPNAVINST  3750.6Q,  1997).  The  use  of  HFACS  in  mishap  reporting  is 
designed  to  improve  the  quality  of  mishap  reports,  which  will  in  turn  improve  the  data 
available  for  analysis.  With  a  higher  quality  database  available,  mishap  analysts  can 
isolate  recurring  causal  factors  and  recommend  suitable  strategies  to  eliminate  or  reduce 
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the  frequency  of  these  accidents.  These  intervention  strategies  can  then  be  evaluated  for 
their  effectiveness  in  improving  operational  readiness  and  reducing  budgetary  losses. 

D.  MISHAP  INTERVENTION  STRATEGIES 

Mishap  intervention  strategies  are  designed  to  address  common  mishap  causes 
and  decrease  the  probability  of  their  occurring.  Typically,  these  strategies  fall  into  one  of 
three  categories:  engineering  controls,  policy  and  procedures  and  individual  protection 
measures.  Engineering  controls  deal  with  a  physical  reconfiguration  of  the  system.  It 
may  be  ergonomic,  or  mechanical  in  nature.  The  result  of  the  change  is  improved  system 
performance.  Policy  and  procedures  address  the  circumstances  surrounding  operations. 
Within  the  HFACS  model,  these  strategies  reflect  mishaps  in  the  unsafe  conditions  and 
unsafe  supervision.  Changes  in  flight  prerequisites  and  improved  time  to  train  are 
examples  of  policy  modifications.  Individual  protection  refers  to  the  equipment  a  person 
may  use  to  decrease  the  possibility  of  physical  injury  due  to  operations.  This  study 
addresses  seven  mishap  intervention  strategies. 

1.  Aeromedical  Screening  and  Education 

Among  the  recommendations  made  by  Schmidt  &  Parker  (1995)  several  are 
associated  with  aeromedical  screening  and  monitoring.  Currently,  this  is  taking  place 
prior  to  assignment  to  UAV  training.  However  once  initial  training  is  complete,  a  flight 
surgeon  is  not  assigned  to  each  UAV  unit.  There  still  exists  a  perception  that  UAV 
crewmembers  are  just  like  any  other  ground  community,  where  crew  rest  and  other 
requirements  are  perceived  as  a  luxury  and  not  a  requirement.  Additionally,  ongoing 
training  is  not  routinely  conducted  to  address  the  affects  of  nutrition,  alcohol  and  tobacco 
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use,  stress  and  psychological  readiness  on  mission  performance.  The  first  intervention 
strategy  addressed  is  continuing  and  increasing  the  education  and  follow  up  attention  to 
aeromedical  issues  and  their  effect  on  readiness.  This  intervention  strategy  is  a  policy  and 
procedure  change  and  is  modeled  by  the  unsafe  condition  (UC)  causation  category. 

2.  Aircrew  Coordination  Training  /  Crew  Resource  Management 

The  policy  and  procedural  recommendation  for  mishap  intervention  is  increased 
aircrew  coordination  training  (ACT)/CRM.  This  is  being  conducted  in  UAV  units; 
however,  the  lack  of  experience  among  crewmembers  limits  its  effectiveness.  Because 
UAV  operators  are  cyclically  new  to  the  community,  there  is  no  senior  leadership  that  can 
speak  with  a  voice  of  experience  to  situations  that  may  occur  during  a  flight.  Aviators  on 
the  crew  bring  a  generic  ACT  background  to  a  unit,  but  lack  specific  UAV  applications. 
This  results  in  a  reactive  rather  than  deliberate  approach  to  an  emergency  situation. 
Improved,  standardized  training  will  impact  mishaps  caused  by  unsafe  acts  (UA)  and 
unsafe  conditions  (UC). 

3.  UAV  Flight  Simulator 

A  UAV  mission  simulator  can  further  enhance  mission  performance  skills  and 
mitigate  unsafe  actions  during  the  conduct  of  a  flight.  Schmidt  &  Parker  (1995)  find  that 
59%  of  mishaps  occurred  as  a  result  of  electrical,  mechanical  or  engine  failure.  If  these 
situations  can  be  replicated  via  simulation,  a  crew  can  rehearse  procedures  to  correct  the 
problem,  or  put  the  aircraft  in  an  attitude  where  damage  is  minimized.  Currently  the 
Pioneer  does  not  have  a  simulation  capability  where  an  instructor  can  induce  such  an 
emergency,  observe  the  crew  response,  and  then  debrief  the  crew  on  their  performance. 
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The  only  time  the  entire  crew  can  operate  as  a  team  to  address  an  emergency  is  during  an 
actual  flight.  A  replacement  to  the  GCS  is  under  development  as  the  Tactical  Control 
System  (TCS).  The  TCS  is  designed  to  be  a  universal  ground  control  and 
communications  shelter  that  has  the  ability  to  interface  with  all  follow-on  DoD  UAV 
systems.  One  of  the  TCS’s  requirements  is  the  ability  to  conduct  simulator  training 
without  conducting  an  actual  mission.  The  model  addresses  unit  training  procedures  and 
is  modeled  in  the  unsafe  act  (UA)  category. 

4.  Automated  Take  off  and  Landing  System 

Schmidt  &  Parker  attribute  32%  of  mishaps  to  take-off  and  landing  error. 
Although  not  currently  a  requirement  for  future  UAV  systems,  several  contractors  are 
developing  an  automated  take-off  and  landing  system.  Their  concept  is  to  fly  the  UAV  to 
a  predetermined  handover  point  where  it  flies  into  a  radio  beacon.  The  beacon  then  takes 
over  sending  navigational  information  to  the  UAV  until  it  is  safely  on  deck.  There  would 
also  be  a  manual  override  system.  This  engineering  control  will  lower  the  incidents  of 
unsafe  acts  (UA)  during  the  takeoff  and  landing  phase  of  the  flight. 

5.  Personnel  Stabilization 

In  order  to  lessen  the  effects  of  unsafe  supervision  as  well  as  unsafe  acts.  The 
UAV  community  can  establish  officer  specialty  codes  and  career  progression 
possibilities.  The  current  situation  of  constantly  rotating  first  time  supervisors  into  UAV 
units  causes  instability  and  limits  the  "corporate  knowledge"  of  squadron  members.  If 
implemented,  supervisory  and  performance  skills  will  dramatically  improve.  The  policy 
change  is  modeled  by  the  unsafe  acts  (UA)  and  unsafe  supervision  (US)  categories. 
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6.  Engine  Replacement 

Although  not  human  factors  related,  engine  failure  has  been  attributed  to  one 
quarter  of  UAV  mishaps  (Schmidt  &  Parker,  1995).  Engineering  modifications  replacing 
the  current  engine  with  a  more  reliable  engine  is  modeled  in  the  analysis  section. 

7.  Electronic  Waterproofing 

Like  engine  failures,  electronic  failures  account  for  another  quarter  of  UAV 
mishaps.  Although  not  originally  designed  as  a  naval  UAV,  the  Pioneer  is  operated  by 
the  U.S.  naval  services  and  would  benefit  by  environmental  shielding.  The  affect  of 
engineering  modifications  such  as  water  proofing  system  components  for  operation  at  sea, 
in  the  littorals,  and  in  foul  weather  is  addressed. 

E.  STOCHASIC  MODELING 

Ross  (1997)  states  that  in  making  a  mathematical  model  for  a  real  world 
phenomenon,  it  is  always  necessary  to  make  simplifying  assumptions  so  as  to  make  the 
mathematics  tractable.  However,  making  too  many  simplifying  assumptions  can  make 
our  conclusions  not  applicable  to  the  real  world.  Therefore,  the  stochastic  model  must 
strike  a  balance  between  simplicity  and  realism. 

Law  &  Kelton  (1991)  state  that  mathematic  model  simulation  is  one  of  the  most 
widely  used  techniques  in  operations  research.  The  mathematical  model  is  used  to 
represent  a  system  in  terms  of  logical  and  quantitative  relationships  that  can  be 
manipulated  to  see  how  the  model  reacts.  A  stochastic  process  is  the  collection  of 
random  variables  ordered  over  time,  which  are  defined  on  a  common  sample  space.  A 
stochastic  simulation  model  takes  random  variable  input  components  and  repeats  the 
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simulation  multiple  times  in  order  to  achieve  a  random,  although  converging  solution. 
The  output  of  a  stochastic  model  is  itself  random,  but  the  number  of  repetitions  can 
decrease  the  variance  in  the  results. 

In  order  to  create  a  stochastic  simulation  model,  probability  distributions  and 
parameters  must  be  identified.  In  lieu  of  using  the  empirical  distribution  that  may  contain 
some  "irregularities,"  particularly  if  the  sample  size  is  small,  a  theoretical,  parametric 
distribution  is  used  to  smooth  out  the  data  (Law  &  Kelton,  1997).  Mintz  (1954a)  models 
taxi  cab  accidents  in  order  to  determine  whether  they  could  be  modeled  by  a  specific, 
known  distribution.  In  order  to  accomplish  this  simplification,  Mintz  makes  two 
assumptions:  (1)  accident  liability  (or  proneness)  of  people  is  not  changed  by  accidents  in 
which  they  are  involved  and  do  not  vary  over  time;  and  (2)  accident  liability  is  distributed 
in  some  known  manner.  Through  his  study  of  over  1200  taxi  cab  accidents,  Mintz 
(1954b)  concludes  that  accident  rates  closely  approximate  a  Poisson  Process  because  (1) 
they  do  display  a  "memory  less"  property;  and  (2)  they  are  distributed  as  an  exponential 
random  variable.  For  the  purpose  of  the  UAV  mishap  model,  parameterized  distributions 
are  tested  to  determine  their  suitability  as  model  inputs. 

The  goal  of  the  stochastic  model  is  to  probabilistically  simulate  annual  Pioneer 
UAV  flight  operations  for  the  Navy  and  Marine  Corps  to  approximate  mishap  events, 
their  cost,  and  effect  on  readiness.  The  model  simulates  these  mishaps  as  a  Poisson 
Process.  The  model  is  designed  with  an  open  architecture  to  model  any  durations  of 
flight  operations,  or  quantify  other  measures  of  performance  (MOPs).  While  it  may  not 
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be  feasible  or  desirable  to  modify  the  Pioneer  UAV  with  these  results,  they  are  applicable 
for  addressing  the  next  generation  replacement  system. 

In  order  to  model  intervention  strategies,  this  model  isolates  causal  factor 
categories.  Although,  none  of  these  intervention  measures  can  be  totally  successful  when 
employed  in  isolation  within  a  mishap  category.  For  example,  ACT  relies  on  experience, 
which  is  tied  to  specific  UAV  occupational  fields  and  career  progression  within  the 
community.  Simulators  also  enhance  training  and  readiness.  Aeromedical  readiness  is 
tied  to  standardization  and  supervision.  All  aspects  of  mishap  prevention  are 
interdependent. 

Stochastic  modeling  is  used  effectively  in  previous  studies  to  effectively  model 
accidents  and  their  effects  on  cost  and  missed  working  hours.  Schmorrow  (1998)  uses 
the  HFACS  taxonomy  to  code  aviation  maintenance  mishaps,  and  stochastic  modeling  to 
predict  cost  and  readiness.  Sciretta  (1999)  uses  a  similar  methodology  to  stochastically 
model  U.S.  Navy  shipboard  electrical  shock  mishaps.  Teeters  (1999),  applying  HFACS, 
studies  the  distribution  of  major  and  minor  aviation  maintenance  mishaps  for  Fleet 
Logistic  Support  (VR)  Wing  aircraft.  This  study  applies  a  similar  framework  to 
stochastically  model  UAV  mishaps. 

F.  SUMMARY 

In  order  to  effectively  analyze  mishap  intervention  techniques,  effective  coding 
and  documentation  is  required.  The  Naval  Aviation  Safety  program  provides  the 
framework  and  resources  to  collect  this  data.  HFACS  is  the  most  recent  improvement  to 
aid  in  accident  investigation,  reporting  and  analysis.  HFACS  is  based  upon  accepted 
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accident  causation  theories.  Using  HFACS  coding,  mishaps  can  be  categorized,  and 
mishap  probabilities  determined.  To  analyze  these  categories,  a  stochastic  model  can  be 
used  to  simulate  mishap  occurrences  over  a  defined  period  of  time.  While  random  in 
themselves,  the  results  are  used  to  weigh  the  various  options  available  to  reduce  mishap 
cost  and  increase  mission  readiness. 

Applying  the  HFACS  taxonomy,  accidents  happen  as  a  result  of  a  confluence  of 
weaknesses  in  all  four  tiers  of  the  model.  Intervention  strategies  are  designed  to  reinforce 
the  cohesion  of  the  tiers  and  decrease  the  probability  of  a  window  of  accident 
opportunity.  Intervention  strategies  are  designed  to  strengthen  safety  environment  at  each 
of  the  four  levels. 
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III.  METHODOLOGY 

A.  RESEARCH  DATA 

The  goal  of  this  research  is  to  use  a  stochastic  model  simulation  to  predict  the 
effects  of  mishap  intervention  strategies  on  both  operational  readiness  and  budgetary 
costs.  The  data  inputs  must  be  determined  in  order  to  conduct  the  simulation.  First,  a 
database  is  constructed  from  the  mishap  reports,  and  causal  factors  are  coded  and  parsed 
from  the  data.  Analysis  of  the  data  leads  to  the  statistical  determination  of  the  model 
inputs:  inter-event  times  for  mishaps;  the  mishap  rate  parameters;  the  probability  of 
mishaps  by  class  (A,  B  or  C);  the  mishap  cost  distribution;  and  the  annual  UAV  flight 
hours.  Once  calculated,  these  statistics  are  used  in  a  stochastic  model  simulation  of 
annual  UAV  flight  operations.  The  output  of  the  model  is  expressed  as  a  mission 
readiness  factor,  and  annual  mishap  costs.  Detailed  description  of  each  phase  is 
discussed  below. 

1.  Mishap  Records 

The  Naval  Safety  Center  (NSC)  is  responsible  for  maintaining  aviation  mishap 
records  for  the  Navy  and  Marine  Corps.  The  foundation  of  the  mishap  record  is  the 
mishap  investigation  report  (MIR).  MIRs  are  required  for  all  Class  A,  B  and  C  mishaps 
in  accordance  with  OPNAVINST  3750.6Q.  The  following  items  are  described  in  the 
MIR:  the  events  leading  up  to  a  mishap,  the  location  and  type  of  operations  involved  in 
the  mishap,  causal  factors  and  recommendations  for  reducing  the  risk  of  similar  type 
accidents  occurring.  Prior  to  FY93,  UAV  mishaps  were  not  incorporated  into  the 
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Aviation  Safety  Program.  As  a  result,  the  MIRs  are  not  complete  and  normally  point  only 
to  one  causal  factor,  usually  mechanical,  electrical  or  human  error.  MIRs  submitted  since 
October  1992  have  improved  mishap  records  significantly.  Primary  and  contributing 
factors  are  reported  with  greater  detail. 

2.  Data  Base 

The  database  for  this  thesis  is  constructed  from  the  UAV  MIRs  maintained  by  the 
NSC,  and  formatted  into  an  EXCEL  spreadsheet.  Each  mishap  event  contains  the 
following  categories:  mishap  date;  air  vehicle  number;  unit;  mishap  summary;  aircraft 
equipment  and  damage;  repair  cost;  time,  location,  altitude,  and  weather  at  the  mishap* 
occurrence;  causal  factors;  and  recommendations.  All  repair  costs  are  converted  into 
FY98  dollars  using  the  aviation  price  inflation  indices  provided  by  the  Naval  Center  for 
Cost  Analysis  (NCC A). 

This  study  addresses  mishap  rates  from  the  entire  database  (FY86-FY98).  When 
detailed  analysis  of  causal  factors  is  required,  it  limits  its  scope  to  the  FY93-FY98  MIRs 
that  are  standardized  by  the  Aviation  Safety  Program.  Additionally,  this  partition  helps  to 
focus  analysis  on  recent  steady  state  UAV  operations.  Initial  mishaps  caused  by  the 
introduction  of  the  air  vehicle  into  the  Fleet  inventory  or  by  the  aberrations  to  normal 
operations  such  as  those  occurring  during  Operations  Desert  Shield  and  Desert  Storm  do 
not  confound  the  data. 

3.  Causal  Factors 

A  single  mishap  typically  has  several  codes  associated  with  it.  This  analysis  goes 
beyond  the  primary  causal  factor,  and  addresses  known  contributing  factors.  Also  of 
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note,  mishap  coding  is  done  at  the  lowest  level  possible  given  the  data  provided.  For 
example,  a  UAI  (unsafe  act:  intended)  is  a  subset  of  UA  (unsafe  act).  However,  if  the 
MIR  provides  limited  information,  coding  is  done  at  the  highest  level  discemable.  The 
UAV  mishaps  are  coded  by  causal  factor  in  accordance  with  the  Human  Factors  Mishap 
Classification  System  (HFACS)  (Shappell  &  Wiegmann,  1997).  Material  failures  are 
recorded  by  causal  factor.  Table  2  contains  the  codes  used  in  the  database: 


CAUSAL  FACTOR 

CODE 

Human  Factors 

HF 

Unsafe  Act 

UA 

Intended 

UAI 

Mistake 

UAIM 

Violation 

UAIV 

Unintended 

UAU 

Slip 

UAUS 

Lapse 

UAUL 

Unsafe  Condition 

UC 

Aeromedical 

UCA 

CRM 

UCC 

Readiness 

ucv 

Unsafe  Supervision 

US 

Unforeseen 

usu 

Foreseen 

USF 

Electro  -  Mechanical 

EM 

Engine 

ENG 

Electrical 

ELEC 

Launcher  Failure 

LNCHR 

Net  Recovery  Failure 

NET 

Software 

SOFT 

Other 

OTHER 

Unknown  or  unspecified 

UNK 

Table  2:  Mishap  Database  Causal  Factor  Codes 
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B.  DATA  ANALYSIS 

1.  Inter-event  Times 

The  computation  of  mishap  inter-event  times  requires  that  a  transformation  be 
made  from  inter-event  days,  as  recorded  by  the  MIR,  to  inter-event  flight  hours,  for  use 
by  the  model.  In  order  to  accomplish  this  transformation,  annual  flight  hours  are  assumed 
to  be  uniformly  distributed  throughout  the  fiscal  year.  An  estimate  is  made  of  daily  flight 
hours  using  annual  flight  hours  flown  information.  This  daily  flight  hour  rate  is 
multiplied  by  inter-event  days  to  transform  inter-event  times  from  days  to  flight  hours 
flown. 

2.  Mean,  Variance,  and  Rate  Parameters 

Unbiased  estimators  for  the  mean  and  variance  of  inter-event  times  are  determined 
for  each  mishap  category  that  is  modeled  by  the  simulation.  The  mishap  rate  (V)  is 
calculated  by  taking  the  inverse  of  the  mean  inter-event  time  (Ross,  1997).  The 
Kolmogorov-Smirnov  goodness  of  fit  (KS  g.o.f.)  test  is  used  to  decide  whether  an 
exponential  distribution  with  parameter  (A,)  is  an  appropriate  model  for  the  inter-event 
data,  and  thus  whether  the  occurrence  of  mishaps  could  be  modeled  as  a  Poisson  Process 
(Conover,  1999).  The  mishap  rate  for  the  entire  data  set  and  the  mishap  rates  by  causal 
factor  are  the  components  of  the  stochastic  counting  process,  which  is  being  modeled. 

3.  Probability  of  Mishap  by  Class 

The  number  of  mishaps  by  class  (A,  B,  C)  is  recorded  for  each  mishap  category. 
The  probability  of  Class  A,  B  and  C  mishaps  is  estimated  by  the  number  of  mishaps  in 
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each  class  divided  by  the  total  number  of  mishaps  in  that  category.  A  vector  of  Class  A, 
Class  B  and  Class  C  probabilities  is  used  as  a  multinomial  input  to  the  model  simulation. 

4.  Cost  Distribution 

A  cost  distribution  of  each  mishap  class  is  determined  by  estimating  the  mean  and 
variance,  and  then  performing  a  K-S  g.o.f.  test  to  confirm  their  suitability.  The  combined 
mishap  rates,  probability  of  accident  severity  and  cost  distributions  are  used  to  describe 
the  model  in  terms  of  a  compound  Poisson  Process. 

5.  Determining  Annual  Flight  Hours 

A  regression  analysis  of  annual  flight  hours  by  fiscal  year  is  used  to  predict  annual 
flight  hours  for  the  stochastic  model.  The  model  uses  the  FY99  flight  hour  prediction  as 
the  input.  The  time  is  the  bound  in  the  simulation  for  the  mishap  generation  period. 

C.  SIMULATING  THE  EFFECTS  OF  MISHAP  INTERVENTION 

The  mishap  simulation  model  inputs  are  the  number  of  simulation  repetitions,  the 
mishap  rate  parameter,  the  multinomial  probability  vector  of  the  mishap  categories  (A,  B 
and  C),  and  the  time  period  to  be  simulated.  The  program  instantiates  two  vectors  to 
store  the  mission  readiness  factor,  and  the  budgetary  cost  value.  Each  simulation  run 
goes  from  time  zero  to  the  end  time  period  input  value.  Time  steps  are  made  as  a  mishap 
is  generated  using  the  exponential  distribution  with  the  appropriate  input  rate  parameter. 
Once  a  mishap  is  encountered,  its  class  is  determined  randomly,  using  the  input 
probability  vector.  Mishap  cost  is  calculated  using  the  predetermined  cost  distribution 
data  for  that  particular  mishap  class  which  is  "hardwired"  into  the  code. 
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The  simulation  run  ends  when  the  time  clock  exceeds  the  input  time  period.  The 
mission  readiness  factor  is  calculated  by  multiplying  the  number  of  class  A  mishaps  by 
three,  multiplying  the  number  of  class  B  mishaps  by  two,  and  adding  in  the  number  of 
class  C  mishaps.  If  this  readiness  factor  exceeds  21  for  the  time  period,  the  mission 
readiness  is  stored  as  a  zero;  otherwise,  the  simulation  returns  a  one  -  mission  ready. 
Both  the  cost  and  mission  readiness  factor  are  stored  in  their  respective  vectors.  Once  the 
simulation  has  completed  the  required  number  of  runs,  the  simulation  returns  the  mean 
and  standard  deviation  of  budgetary  cost  (FY98$M),  and  the  average  of  the  readiness 
factor.  The  mishap  intervention  model  code,  MishapSim()  is  programmed  in  S-Plus  4.0 
and  is  attached  as  Appendix  B. 

For  each  mishap  intervention  strategy,  the  model  is  run  one  thousand  repetitions 
and  through  a  total  of  four  simulations.  The  first  simulation  is  baseline  simulation  using 
the  current  mishap  rate.  A  reduction  of  the  mishap  rate  by  10%,  30%  and  50%  for  each 
intervention  strategy  is  hypothesized  for  the  next  three  simulation  runs.  If  two  or  more 
mishap  categories  are  being  modeled  together,  for  example,  unsafe  acts  and  unsafe 
supervision,  the  rate  parameters  are  determined  as  described  above.  However,  all 
replicated  mishaps  are  removed  from  the  composite  mishap  category.  These  calculations 
will  enable  fleet  users  and  program  managers  to  weigh  the  effectiveness  of  a  proposed 
change  with  the  resulting  cost  and  readiness  savings. 
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IV.  RESULTS 

A.  OVERVIEW 

This  chapter  presents  a  database  summary  for  annual  flight  operations  by  mishap 
class,  and  flight  hours  flown.  Mishap  causal  frequencies  are  summarized  for  operations 
going  back  to  1986,  and  for  the  period  of  the  study,  1993-98.  Mishap  coding  results  are 
presented  by  both  number  and  frequency.  All  simulation  model  input  parameters  are  then 
estimated  and  presented.  Simulation  results  for  the  each  mishap  intervention  strategy  are 
presented  for  the  existing  baseline  and  for  mishap  frequency  reductions  of  10  percent,  30 
percent  and  50  percent.  The  chapter  concludes  with  a  graphic  comparison  of  cost  and 
readiness  results  for  all  intervention  categories. 

B.  BACKGROUND  UAV  FLIGHT  DATA 

Figure  7  is  a  graph  of  flight  hours  flown  and  the  number  of  mishaps  versus  time. 
Table  3  contains  a  summary  of  the  UAV  annual  flight  hours,  mishaps  and  associated  rates 
for  the  period  since  the  fielding  of  the  Pioneer  system  by  the  Navy  and  Marine  Corps  in 
1986.  In  general,  the  annual  flight  hours  flown  is  increasing  and  the  mishap  rate  is 
decreasing.  During  Operation  Desert  Shield  and  Desert  Storm,  six  UAV  units  flew  just 
under  1000  flight  hours.  These  units  deployed  with  thirty  air  vehicles.  At  current  mishap 
rates  and  during  a  deployment  of  equal  duration,  approximately  one  third  of  the  air 
vehicles  would  be  destroyed  or  damaged  by  flight  mishaps.  This  prediction  demonstrates 
unacceptable  mission  readiness  and  strains  maintenance  and  repair  capabilities. 
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Fiscal  Year 


C.  MISHAP  CODING 


The  frequency  of  mishap  cause  occurrences  is  identified  in  Table  4.  This  data  is 
consistent  with  the  previous  studies  of  Schmidt  &  Parker  (1995)  and  Seagle  (1997). 
Appendix  C  contains  the  complete  mishap-coding  database.  Of  note,  the  frequency  of 
human  factors  related  mishaps  is  not  increasing.  Rather,  the  reporting  of  human  factors 
related  mishaps  is  increasing  in  detail.  The  Mishap  Investigative  Reports  (MIRs) 
submitted  since  FY  93  have  increased  the  information  available  upon  which  to  assign 
causal  factors.  Table  5  is  a  summary  of  each  category  of  mishap  causation.  The 
simulation  model  evaluates  seven  mishap  intervention  strategies.  Each  intervention 


strategy  and  its  associated  mishap  classification  category  are  identified  in  Table  6. 


[Mishaps 

Class  A 

Class B 

Class  C 

Total 

Percentage  | 

|FY  86-98  j 

Overall 

67 

48 

108 

223 

100% 

Human  Factors 

15 

24 

48 

87 

39% 

Electro-Mechanical 

55 

30 

65 

150 

67% 

|FY  93-98  | 

Overall 

24 

31 

38 

93 

100% 

Human  Factors 

11 

20 

24  . 

55 

59% 

Electro-Mechanical 

17 

18 

21 

56 

60% 

Table  4:  Mishap  Causation  Frequency 


51 


1 _ _ 

FY  86-98  |  FY  93-98  | 

CAUSAL  FACTOR 

CODE 

# 

FREQ 

# 

Human  Factors 

ai 

87 

39.0% 

55 

59.1% 

Unsafe  Act 

mm 

35 

15.7% 

35 

37.6% 

Intended 

UAI 

16 

7.2% 

16 

17.2% 

Mistake 

UAIM 

11 

4.9% 

!  11 

11.8% 

Violation 

UAIV 

6 

2.7% 

6 

6.5% 

Unintended 

UAU 

19 

8.5% 

19 

20.4% 

Slip 

UAUS 

13 

5.8% 

!  13 

14.0% 

UAUL 

3 

1.3% 

3 

3.2% 

Unsafe  Condition 

37 

16.6% 

37 

39.8% 

Aeromedical 

UCA 

9 

4.0% 

9 

9.7% 

CRM 

ucc 

26 

11.7% 

26 

28.0% 

Readiness  Violation 

ucv 

9 

4.0% 

9 

9.7% 

Unsafe  Supervision 

US 

40 

17.9% 

40 

43.0% 

Unforeseen 

usu 

14 

14 

15.1% 

Foreseen 

USF 

11 

4.9% 

11 

11.8% 

Electro  -  Mechanical 

EM 

158 

70.9% 

64 

68.8% 

Engine 

52 

23.3% 

23 

24.7% 

Electrical 

ELEC 

59 

26.5% 

20 

21.5% 

Launcher  failure 

LNCHR 

8 

3.6% 

2 

2.2% 

Net  recovery  failure 

NET 

16 

7.2% 

7 

7.5% 

Software 

SOFT 

7 

3.1% 

5 

5.4% 

Other 

OTHER 

20 

9.0% 

8 

8.6% 

Unknown/  unspecified 

[3391 

8 

3.6% 

7 

7.5% 

Table  5:  Mishap  Frequency  by  Causation  Code 
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Mishap  Intervention  Strategy 

Mishap  Category 

Aeromedical  Screening  and  Education 

Unsafe  Conditions  (UC) 

Aircrew  Coordination  Training  /  Crew 
Resource  Management 

Unsafe  Acts,  Unsafe  Conditions  (UA/UC) 

Flight  Simulator  /  TCS 

Unsafe  Acts  (UA) 

Automatic  Takeoff  and  Landing  System 

Unsafe  Acts  (UA) 

Personnel  Stabilization 

Unsafe  Acts,  Unsafe  Supervision  (UA/US) 

Engine  Upgrade 

Engine  (ENG) 

Weatherizing  the  Air  vehicle 

Electronic  (ELEC) 

Table  6:  Mishap  Intervention  Strategy  and  Associated  Causal  Category 


D.  STOCHASTIC  MISHAP  MODEL  ESTIMATES 

1.  Inter-event  Times 

Figures  8  through  13  are  histogram  plots  of  the  inter-arrival  times  for  each 
partition  of  mishap  causal  factors.  Overlaid  on  the  chart  is  a  rescaled  probability  density 
function  (pdf)  of  the  hypothesized  exponential  distribution.  Table  7  summarizes  the 
mean,  standard  deviation  and  rate  parameters  for  the  mishap  category  parameter 
estimates.  Additionally,  95%,  two-sided  confidence  intervals  are  presented  for  each  rate 
parameter  estimate.  Note  that  the  two  confidence  limits  are  not  equidistant  from  the  point 
estimate.  This  is  due  to  the  lack  of  symmetry  in  the  exponential  distribution.  The 
Kolomogorov-Smimov  goodness  of  fit  (KS  g.o.f.)  test  for  the  exponential  distribution  is 
based  on  the  estimated  rate  parameter.  With  a  significance  level  (a)  set  at  0.05,  the  KS 
g.o.f.  test  fails  to  reject  that  any  of  the  distributions  are  exponential. 


53 


Figure  9:  Histogram  Plot  of  UA/UC  Data 
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Figure  1 1 :  Histogram  Plot  of  UA/US  Data 
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Estimated 

Mean 

Estimated 
Std  Dev 

Estimated 

Rate 

Lower 

Cl 

Upper 

Cl 

KS  g.o.f 
test 

Category 

n 

flight  hours 

flight  hours 

p-value 

UC 

37 

238.6 

216.2 

4.19 

2.95 

5.65 

0.665 

UA/UC 

47 

195.3 

189.8 

5.12 

3.76 

6.68 

0.406 

UA 

35 

262.3 

212.7 

3.81 

2.66 

5.18 

0.160 

UA/US 

52 

176.5 

156.9 

5.67 

4.23 

7.31 

0.484 

ENG 

23 

406.9 

479.5 

2.46 

1.56 

3.56 

0.478 

ELEC 

20 

467.6 

453.8 

2.14 

1.31 

3.17 

0.409 

Table  7:  Parameter  Estimates  for  Mean,  Standard  Deviation  and  Rate 


2.  Mishap  Class  Probability  Parameters 

Table  8  is  a  summary  of  the  calculations  used  to  estimate  the  probability  of  each 
type  of  mishap  (pA,  Pb,  Pc)-  Of  note,  the  estimated  probability  of  each  class  of  mishap 
(pA-Hat,  PB-Hat*  Pc-Hat)  varies  with  the  mishap  category.  For  example,  an  engine  failure  is 
most  likely  to  cause  a  catastrophic  class  A  mishap,  while  the  results  of  an  unsafe  act  can 
be  mitigated  by  other  actions  of  the  crew. 


Class  A 

Class B 

Class  C 

Code 

n 

#  mishaps 

PA-Hat 

#  mishaps 

PB-Hat 

#  mishaps 

PC-Hat 

a 

37 

6 

0.162 

16 

0.432 

15 

0.405 

UA/UC 

47 

8 

0.170 

19 

0.404 

20 

0.426 

HR 

35 

5 

0.143 

15 

0.429 

15 

0.429 

UA/US 

52 

10 

0.192 

20 

0.385 

22 

0.423 

ENG 

23 

9 

0.391 

4 

0.174 

10 

0.435 

20 

4 

0.200 

10 

0.500 

6 

0.300 

Table  8:  Mishap  Class  Probabilities 
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3. 


Cost  Parameters 


Table  9  summarizes  the  parametric  estimates  for  the  mishap  cost  distributions. 
Each  is  hypothesized  to  be  normally  distributed.  All  cost  figures  are  calculated  as  FY98 
dollars.  Figures  14  through  16  are  normal  probability  plots  of  the  actual  data  versus  the 
hypothesized  distributions.  Graphically,  there  are  some  discrepancies  between  the  sample 
data  and  the  hypothesized  distribution.  However,  the  KS  g.o.f.  tests  the  hypothesized 
distribution  using  the  parameter  estimates  listed  in  Table  9.  With  a  significance  level  (a) 
set  at  0.05,  the  KS  g.o.f.  test  fails  to  reject  the  normality  of  any  of  the  cost  distributions. 


Estimates 

Distribution 

KS  g.o.f. 

Mean 

Std  Dev 

$  FY98 

p-value 

Class  A 

$811,504 

$189,306 

N(812K,  187K) 

0.4802 

Class  B 

$479,933 

$214,503 

N(480K,  214K) 

0.6073 

Class  C 

$87,649 

$64,065 

N(88K,  64K) 

0.1928 

Table  9:  FY86-FY98  Mishap  Class  Cost  Distribution 
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Normal  Quantiles 


Figure  16:  Normal  Probability  Plot  of  Class  C  Mishap  Costs 

4.  Annual  Flight  Hours 

A  linear  regression  of  annual  flight  hours  versus  the  natural  log  of  fiscal  year  is 

performed  on  historic  flight  hours  flown.  The  equation-projected  estimate  of  flight  hours 

< 

for  the  next  fiscal  year  (FY99)  is  1,930  flight  hours.  This  will  be  used  as  the  number  of 
annual  flight  hours  in  the  simulation.  Figure  17  contains  a  graph  of  the  historic  flight 
hours  and  the  fitted  equation,  adjusted  to  time  on  a  linear  scale. 
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Figure  17:  Results  of  Annual  Flight  Hour  Regression 

E.  STOCHASTIC  MODEL  SIMULATION 
1.  Model:  Baseline  of  total  mishaps 

The  aggregate  mishap  model  is  presented  in  Table  10.  It  is  used  as  a  baseline  for 
comparison  of  the  remaining  models.  Calculating  the  defined  readiness  factor,  this  model 
indicates  that  UAVs  never  achieve  a  mission  ready  condition.  Also,  UAV  mishap  costs 
typically  exceed  $10  million.  The  following  model  simulations  are  used  to  gain  insight 
into  the  cost  and  mission  readiness  improvements  over  current  baseline  conditions  made 
by  targeting  mishap  causes  with  the  specified  strategies. 


61 


Category:  TOTAL 

Baseline 

X  - 10% 

X  -  30% 

X  -  50% 

Mishap  Rate  / 1000  flight  hours 

12.79 

11.51 

8.95 

6.4 

Mean  Cost  (CY98  $M) 

$10.91 

$9.70 

$7.54 

$5.63 

SD  Cost  (CY98  $M) 

$2.83 

$2.62 

$2.30 

$1.92 

%  change  in  Cost 

IKS* 

-11.2% 

-30.9% 

-48.4% 

Readiness  Index 

0.0000 

0.0020 

0.0640 

0.3000 

%  change  in  Readiness 

M 

N/A 

N/A 

N/A 

Table  10:  Aggregate  Mishap  Model 


2.  Model:  Increased  Aeromedical  Screening  and  Education 
Table  1 1  summarizes  the  unsafe  conditions  mishap  intervention  model.  Even  at 
current  levels,  the  readiness  indicator  for  unsafe  conditions  does  not  go  below  80%.  This 
is  most  likely  the  result  of  the  relative  probability  of  a  class  A  mishaps  being  low 

compared  to  the  aggregate  model.  As  a  result,  costs  are  also  kept  low,  accounting  for 

( 

approximately  1/3  of  the  aggregate  mishap  costs. 


Category:  UC 

Baseline 

X- 10% 

X  -  30% 

X  -  50% 

Mishap  Rate  / 1000  flight  hours 

4.20 

3.78 

2.95 

2.10 

Mean  Cost  (CY98  $M) 

$3.51 

$3.09 

$2.60 

$1.94 

SD  Cost  (CY98  $M) 

$1.44 

$1.36 

$1.21 

$1.03 

%  change  in  Cost 

- 

-12.1% 

-•26.1% 

-44.8% 

Readiness  Index 

0.8400 

0.9170 

0.9710 

0.9970 

%  change  in  Readiness 

■ 

IppiajjjtKg 

9.2% 

15.6% 

18.7% 

Table  1 1 :  Unsafe  Conditions  Model 
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3.  Model:  Aircrew  Coordination  Training  /  Crew  Resource  Management 
Table  12  summarizes  the  mishap  intervention  model  for  the  aggregate  of  unsafe 
acts  and  unsafe  conditions.  ACT  /  CRM  will  have  to  reduce  the  mishap  rate  by  over  10% 
in  order  to  get  readiness  above  the  80%  level.  Additionally,  at  $4  million  per  year  in 
mishap  cost  contributions,  unsafe  acts  and  unsafe  conditions  combined  contribute  to 
approximately  one-third  of  mishap  costs. 


Category:  UA/UC 

Baseline 

X  - 10% 

X  -  30% 

X  -  50% 

Mishap  Rate  / 1000  flight  hours 

5.11 

4.60 

3.58 

2.56 

Mean  Cost  (CY98  $M) 

$4.01 

$3.63 

$2.93 

$2.16 

SD  Cost  (CY98  $M) 

$1.56 

$1.40 

$1.29 

$1.10 

%  change  in  Cost 

I . s® 

-9.5%. 

-26.8% 

-46.0% 

Readiness  Index 

0.7040 

0.8010 

0.9350 

0.9910 

%  change  in  Readiness 

7 m. 

13.8% 

32.8% 

40.8% 

Table  12:  Aggregate  Unsafe  Acts  /  Unsafe  Conditions  Model 


4.  Model:  UAV  Flight  Simulator  /  Automated  Take  off  and  Landing 
System 

Table  13  summarizes  the  mishap  intervention  model  for  unsafe  acts.  This  model 
is  used  to  evaluate  the  potential  results  of  both  the  flight  simulator  and  of  the  take  off  and 
landing  aids.  While  the  category  of  unsafe  acts  contributes  to  37%  percent  of  mishaps, 
their  effect  on  readiness  is  not  as  profound.  Even  in  the  baseline  case,  unsafe  acts  have  a 
ready  index  of  nearly  90  percent.  Because  of  their  nature,  unsafe  acts  will  contribute  to 
mishap  occurrences,  but  their  individual  effects  do  not  have  as  great  an  impact  on  overall 
changes  in  readiness  and  costs. 
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Category:  UA 

Baseline 

X-10% 

X  -  30% 

X-50% 

Mishap  Rate  / 1000  flight  hours 

3.82 

3.44 

2.67 

1.91 

Mean  Cost  (CY98  $M) 

$3.16 

$2.94 

$2.29 

$1.74 

SD  Cost  (CY98  $M) 

$1.40 

$1.29 

$1.11 

$0.95 

%  change  in  Cost 

-6.9% 

-27.6% 

-44.8% 

Readiness  Index 

0.8970 

0.9430 

0.9860 

1.0000 

%  change  in  Readiness 

5.1% 

9.9% 

11.5% 

Table  13:  Unsafe  Acts  Model 


5.  Model:  Personnel  Stabilization 

Table  14  summarizes  the  mishap  intervention  model  for  the  aggregate  of  unsafe 
acts  and  unsafe  supervision.  This  is  the  only  model  that  incorporates  the  effects  of  unsafe 
supervision.  Although  difficult  to  isolate,  unsafe  supervision  when  coupled  with  unsafe 
acts  does  have  a  profound  effect  on  both  cost  and  readiness.  The  model  bears  out  that 


personnel  stability  has  the  potential  to  significantly  improved  readiness  and  reduced  cost. 


Category:  UA/US 

Baseline 

X  - 10% 

X  -  30% 

X  -  50% 

Mishap  Rate  / 1000  flight  hours 

5.66 

5.09 

3.96 

2.83 

Mean  Cost  (CY98  $M) 

$4.52 

$4.20 

$3.39 

$2.52 

SD  Cost  (CY98  $M) 

$1.76 

$1.60 

$1.44 

$1.21 

%  change  in  Cost 

3'  :  ’  : 

-7.2% 

-25.2% 

-44.4% 

Readiness  Index 

0.5570 

0.6760 

0.8710 

0.9750 

%  change  in  Readiness 

— 

21.4% 

56.4% 

75.0% 

Table  14:  Aggregate  Act/Unsafe  Supervision  Model 
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6.  Model:  Engine  Replacement 

Table  15  summarizes  the  mishap  intervention  model  for  engine  failure  related 
events.  Engine  failure  accounts  for  nearly  25  percent  of  mishaps.  Intuitively, 
modifications  to  the  engine  should  cause  a  significant  cost  reduction  and  readiness 
improvement.  However,  the  model  results  indicate  that  the  effect  on  mission  readiness  is 
negligible  since  the  baseline  readiness  factor  already  exceeds  95  percent.  Additionally, 
the  budgetary  cost  of  these  mishaps  is  less  than  or  equal  to  other  mishap  categories. 


Category:  ENG 

Baseline 

X  - 10% 

X  -  30% 

X  -  50% 

Mishap  Rate  / 1000  flight  hours 

2.46 

2.21 

1.72 

1.23 

Mean  Cost  (CY98  $M) 

$2.64 

$2.44 

$1.90 

$1.49 

SD  Cost  (CY98  $M) 

$1.29 

$1.28 

$1.15 

$0.92 

%  change  in  Cost 

-7.6% 

-28.0% 

-43.6% 

Readiness  Index 

0.9750 

0.9780 

0.9990 

1.0000 

%  change  in  Readiness 

0.3% 

2.5% 

2.6% 

Table  15:  Engine  Model 


7.  Model:  Electronic  Waterproofing 

Table  16  summarizes  the  mishap  intervention  model  for  the  improvements  to 
electronic  component  reliability.  Electronic  component  failure  is  cited  in  over  20%  of 
UAV  mishaps,  but  similar  to  engine  failure,  its  impact  on  readiness  and  annual  costs  is 
overshadowed  by  the  other  causal  categories.  The  model  indicates  that  the  effect  of 
electrical  failures  upon  mission  readiness  is  minimal. 
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Category:  ELEC 

Baseline 

a. -10% 

X-30% 

X  -  50% 

Mishap  Rate  / 1000  flight  hours 

2.14 

1.93 

1.50 

1.07 

Mean  Cost  (CY98  $M) 

$2.20 

$2.06 

$1.70 

$1.34 

SD  Cost  (CY98  $M) 

$1.07 

$1.01 

$0.91 

$0.83 

%  change  in  Cost 

P25 ! 

-6.6% 

-23.0% 

-39.2% 

Readiness  Index 

0.9980 

0.9980 

1.0000 

1.0000 

%  change  in  Readiness 

0.0% 

0.2% 

0.2% 

Table  16:  Electronic  Model 


F.  MODEL  COMPARISON 

Figures  18  and  19  are  a  comparison  between  the  results  of  the  mishap  intervention 
simulations.  Figure  18  compares  the  cost  reduction  in  each  category  for  the  four  runs  of 
the  model.  Clearly,  there  is  improvement  across  all  strategies,  but  improvements  in 
UA/UC  and  UA/US  appear  to  cause  more  significant  cost  savings.  Figure  19  illustrates 
the  mission  readiness  index  improvement.  Inspection  of  the  graph  reveals  that  addressing 
the  UC,  UA/UC  and  UA/US  categories  will  have  the  greatest  impact  on  mission 
readiness.  According  to  the  simulation,  none  of  other  causal  factors  reduces  the  readiness 
factor  below  80  percent. 
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□  Baseline  MX- 10%  CU- 30%  □  X - 50% 


TOTAL  UC  UA/UC  UA  UA/US  ENG  ELEC 


Mishap  Category 


Figure  18:  Mishap  Cost  Reduction 


□  Baseline  mX- 10%  DX-30%  DX-50% 


Mishap  Category 


Figure  19:  Mishap  Readiness  Index  Improvement 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  MISHAP  CLASSIFICATION 

Unmanned  Aerial  Vehicle  (UAV)  mishaps  are  successfully  categorized  by  the 
Human  Factors  Accident  and  Classification  System  (HFACS)  using  the  Naval  Aviation 
Safety  Program  (OPNAVINST  3750.6Q)  mishap  report  procedures.  However, 
granularity  at  the  lowest  tiers  of  the  HFACS  model  can  be  difficult  to  ascertain.  The 
information  available  for  analysis  appears  to  vary  with  the  writer  of  the  report.  If  that 
individual  is  aware  and  comfortable  with  HFACS,  the  report  tends  to  address  all  of  the 
details  required  for  future  analysis.  Otherwise,  reports  contain  generalities  and  lack  the 
specific  details  necessary  to  conduct  an  in  depth  analysis  of  the  data.  The  planned 
incorporation  of  the  HFACS  taxonomy  into  the  next  revision  of  OPNAVINST  3750.6R, 
should  standardize  reporting  procedures,  and  educate  those  preparing  mishap  reports  on 
the  scope  that  human  factors  can  have  on  flight  operations. 

This  analysis  of  mishap  classification  is  conducted  independently  of  the  data 
partition  of  Schmidt  &  Parker  (1995)  and  Seagle  (1997).  The  three  separate  mishap 
categorizations  are  similar  in  terms  of  the  relative  frequency  and  causes  of  mishaps. 
Individual  judgments  were  required  by  those  conducting  the  study  in  order  to  place  a 
particular  mishap  into  a  causation  category.  While  not  always  agreeing  in  the  exact 
classification,  the  three  studies  do  conclude  that  human  error  is  at  least  partially 
attributable  for  approximately  one  half  of  UAV  mishaps.  Engine  and  electronic  failure 
each  account  for  20  to  25  percent  of  mishaps.  Finally,  although  the  mishap  rate  is 
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decreasing,  current  mishap  damage  and  losses  will  continue  to  have  a  profound  affect  on 
mission  readiness. 

B.  MISHAP  MODELING 

The  mishap  occurrence  distribution  is  modeled  effectively  as  a  Poisson  process. 
Cost  data  and  mishap  class  data  distributions  can  also  be  modeled  by  specific 
distributions.  These  distributional  quantities  are  effectively  input  into  the  stochastic 
model,  allowing  the  analyst  to  simulate  annual  UAV  flight  operations  accurately. 

C.  MODEL  RESULTS 

The  aggregate  UAV  mishap  rate  must  be  reduced  by  one  half  in  order  to  achieve  a 
significant  change  in  total  mishap  occurrences.  This  mishap  reduction  still  only  raises  the 
readiness  index  to  0.300.  (See  Figure  19.)  In  order  to  focus  the  mishap  intervention 
strategies,  they  must  be  analyzed  in  isolation.  Under  the  current  baseline  rates  two 
mishap  categories,  Unsafe  Acts  and  Unsafe  Conditions  (UA/UC),  and  Unsafe  Acts  and 
Unsafe  Supervision  (UA/US),  cause  the  readiness  index  to  fall  below  0.800.  The 
simulations  demonstrate  that  improvements  up  to  40.8  percent  and  75.0  percent, 
respectively,  can  be  achieved  in  these  categories.  This  indicates  that  they  should  be 
considered  primary  targets  for  intervention  strategies. 

The  other  impact  of  the  simulated  UAV  mishaps  is  cost.  Again,  the  UA/UC  and 
UA/US  categories  are  the  two  most  costly  mishap  causal  factors.  Each  contributes 
greater  than  $4  million  (FY98$)  to  annual  mishap  costs  in  the  model.  Intervention  in 
these  categories  can  reduce  costs  by  44.4  percent  and  40.8  percent,  respectively. 
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Although  not  considered  in  this  model,  intervention  strategies  should  be  compared  to  find 
the  best  value  for  the  money  invested. 

Contrary  to  expectations,  incorporating  engineering  modifications,  (engine 
improvement/replacement,  and  electronic  waterproofing)  will  have  marginal  effects  on 
readiness  and  cost.  At  current  mishap  rates,  the  engine  and  electronic  configuration  do 
not  degrade  mission  readiness  below  95  percent.  Additionally,  their  mishap  costs  are 
approximately  the  same  as  the  other  mishap  categories.  The  cost  involved  in  research, 
development  and  procurement  will  most  likely  exceed  current  mishap  cost  predictions 
and  can  be  better  spent  improving  other  aspects  of  the  Pioneer  system. 

D.  RECOMMENDATIONS 

The  evaluation  of  the  stochastic  model  results  points  to  mishap  intervention 
measures  for  the  UA/UC  and  UA/US  categories.  This  research  proposes  improvements 
to  aircrew  coordination  training  and  crew  resource  management  in  order  to  alleviate  the 
effects  of  the  UA/UC  category.  Initiatives  in  this  area  will  not  only  improve  Pioneer 
UAV  operations,  but  will  have  a  listing  impact  on  the  UAV  community,  regardless  of  the 
system  employed.  When  the  Pioneer  is  eventually  replaced,  only  minor  modifications 
should  be  necessary  to  adjust  to  the  idiosyncrasies  of  the  new  systems.  The  second  area 
of  intervention  recommended  is  UAV  personnel  stabilization.  The  UAV  community  is 
still  relatively  new  to  Naval  Aviation  operations.  As  such,  the  community  needs  to 
mature.  Unit  leaders,  both  officer  and  enlisted,  should  have  experience  and  knowledge  of 
the  system  in  order  to  effectively  manage  unit  operations  and  individual  crewmembers.  A 
UAV  career  path  should  be  created  to  track  these  individuals  and  assign  them 
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appropriately  throughout  their  careers  so  that  the  UAV  community  can  benefit  from  the 
stability  and  "corporate  knowledge"  enjoyed  by  other  Naval  flight  communities. 
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APPENDIX  A:  PIONEER  UAY  SYSTEM  DESCRIPTION 

A.  SYSTEM  COMPONENTS 

A  Pioneer  UAV  system  consists  of  five  Pioneer  air  vehicles,  a  ground  control 
station  (GCS),  a  portable  control  station  (PCS),  a  tracking  communications  unit  (TCU),  a 
data  link,  two  remote  receiver  stations  (RRS)  and  a  reconnaissance  payload.  The  system 
can  be  operated  aboard  specially  configured  USS  Austin  Class  Landing  Platform  Dock 
(LPD-4)  ships  or  from  prepared  airstrips  ashore. 

1.  The  Air  Vehicle 


Figure  20:  pioneer  Air  vehicle 


The  Pioneer  vehicle  air  is  14  feet  long  and  is  pusher-propeller  driven,  powered  by 
a  26  hp,  two  stroke,  twin  cylinder,  rear  mounted  engine,  similar  to  a  snowmobile  engine. 
The  air  vehicle  is  made  of  fiberglass,  Kevlar  and  other  low  cost  composite  materials,  and 
weighs  463  lbs.  The  air  vehicle  can  operate  up  to  an  altitude  of  15,000  feet,  but  normally 
flies  between  3000  and  5000  feet  in  order  to  optimize  payload  performance.  Because  the 
air  vehicle  uses  a  laminated  wood  propeller,  and  the  electronic  components  are  not 
weatherized,  the  UAV  cannot  fly  through  visible  moisture  (fog,  clouds,  rain,  etc.)  or  icing 
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conditions.  The  air  vehicle  has  up  to  four-hour  time  of  flight  at  a  cruise  speed  of  65 
knots,  which  normally  translates  into  2-2.5  hours  in  an  objective  area  depending  on  the 
proximity  of  the  airstrip  to  the  objective. 

The  air  vehicle  is  launched  using  one  of  three  methods.  The  rocket-assisted  take 
off  (RATO)  is  the  only  method  available  for  shipboard  operations.  A  rocket  is  placed 
under  the  vehicle  to  propel  it  into  the  air.  Having  reached  sufficient  altitude  and  airspeed, 
the  rocket  motor  shuts  down  and  is  jettisoned  from  the  UAV.  Land-based  units  can  also 
conduct  RATO  launch.  Additionally  on  land,  the  UAV  can  use  a  standard  rolling  take  of 
from  a  1500-foot  runway.  Because  of  restrictive  crosswind  parameters,  or  air  density 
constraints,  a  rolling  takeoff  may  not  be  possible.  For  these  instances,  a  pneumatic 
launcher  mounted  on  a  5-ton  truck  can  propel  the  vehicle  to  the  minimum  altitude  and 
airspeed  to  transition  to  vehicle-powered  flight. 


figure  21:  RATO  Takeoff 
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Figure  22:  Pneumatic  Launcher  Takeoff 


There  are  two  ways  to  safely  recover  the  air  vehicle.  Operations  at  sea  require  the 
UAV  to  be  flown  into  a  large  net  suspended  across  the  aft  part  of  a  ship’s  helicopter  flight 
deck.  Once,  in  the  net,  the  recovery  system  collapses  around  the  UAV  allowing  it  to  be 
lifted  out.  Because  this  recovery  method  is  tantamount  to  a  controlled  crash,  there  is 
frequent  damage  to  the  UAV.  The  second  recovery  method,  used  ashore,  is  an  arrested 
recovery  by  a  miniature  tailhook  on  an  airstrip.  While  much  more  suitable  for  a  crash 
free  recovery,  cross  wind  limitation  must  be  monitored  in  order  to  assure  a  successful 
recovery  (MAWTS-1, 1997). 


Figure  23:  UAV  Shipboard  Landing 


2.  The  Ground  Control  Station  (GCS) 

The  GCS  is  the  focus  of  activity  for  UAV  missions.  The  system  can  either  be 
land  based  or  installed  aboard  ship.  The  GCS  consists  of  three  electronics  bays  manned 
by  two  operators.  The  pilot  bay  includes  all  controls,  instruments  and  displays  required  to 
safely  "fly"  the  vehicle.  The  observer  bay  provides  control  and  display  of  the  imaging 
payloads  carried  by  the  UAV.  The  tracking  bay  displays  the  UAV  position  based  on  data 
from  the  TCU  and  global  positioning  satellites  (GPS)  (Pioneer  UAV,  INC.,  1999). 


Figure  24:  Inside  the  GCS 


3.  The  Portable  Control  Station  (PCS) 

The  PCS  provides  the  capability  to  control  the  UAV  during  pre-flight,  launch  and 
recovery  operations,  allowing  the  GCS  to  locate  where  it  can  most  effectively  conduct  the 
mission.  Because  the  air  vehicle  relies  on  line  of  site  communications  between  the 
control  station  and  the  air  vehicle,  split  sight  operations  are  common  in  rugged, 
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compartmented  terrain.  The  PCS  provides  the  ability  to  control  the  launch  sequence  from 
a  local  airstrip,  and  then  steer  the  air  vehicle  to  a  predetermined  handover  point.  There, 
the  GCS,  operating  from  a  more  advantageous  location,  can  take  control  of  the  UAV  and 
conduct  the  mission  further  down  range  (MAWTS-1, 1997). 


FIGURE  25 :  INSIDE  THE  PCS  CONTROL  BAY 

4.  The  Tracking  Control  Unit  (TCU) 

The  TCU  shelter  contains  the  UAV  communication  equipment  and  antennas.  The 
TCU  contains  a  sophisticated,  jam  resistant,  C-band,  100  nmi.  range  data  link.  Both  the 
video  and  telemetry  link  use  directional  antennas  between  the  air  vehicle  and  the  TCU  in 
order  to  ensure  video  quality  and  minimize  the  probability  of  data  link  intercept  by  the 
enemy.  The  system  also  has  an  omni-directional,  UHF  backup  link  for  redundancy.  The 
TCU  can  be  remoted  1000  meters  from  the  GCS  by  fiber-optic  cable,  enhancing  the 
system’s  and  personnel  battlefield  survivability  (Pioneer  UAV,  INC.,  1999). 
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FIGURE  26:  THE  TCU 


5.  Remote  Receiving  Station  (RRS) 

The  ruggedized  RRS  provides  real-time  reception  of  the  UAV  video  picture  at 
remote  locations.  The  Marine  Corps  has  mounted  the  RRS  on  a  high  mobility  multi¬ 
wheeled  vehicle  (HMMWV),  a  light  armored  vehicle  (LAV)  and  aboard  a  UH-1N  Huey 
helicopter,  allowing  the  tactical  commander  to  have  real-time  imagery  regardless  of 
where  the  command  post  is  locating  (MAWTS-1, 1997). 


FIGURE  27:  THE  RRS 


6.  Reconnaissance  Payloads 

The  air  vehicle  can  carry  one  of  two  separate,  gyrostabilized  payloads:  the  MKD- 
200  electro-optical  day  camera,  and  the  MKD-400(C)  forward-looking  infrared  (FLIR) 
night  camera.  The  MKD-200  E-0  camera  can  detect  targets  up  to  18  km  range,  and 
recognizes  targets  at  3  km  range.  The  MKD-400(C)  FLIR  camera  can  detect  a  target  at  8 
km  range,  and  recognizes  a  target  at  up  to  4  km.  Camera  performance  is  enhanced  by 
increased  thermal  differential  between  the  target  and  the  surrounding  background 
(MAWTS-1, 1997). 


FIGURE  28:  UAV  PAYLOADS 


B.  CREW  COMPOSITION 

The  term  "unmanned"  is  actually  a  misnomer  when  applied  to  the  Unmanned 
Aerial  Vehicle  system  because  UAV  operations  involve  many  participants.  The  essential 
members  of  a  UAV  crew  include  a  Mission  Commander,  an  Internal  Pilot,  a  Payload 
Operator  and  an  External  Pilot.  Additionally  launch  and  recovery  teams  and  maintenance 
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personnel  will  be  involved  in  flight  operations.  The  responsibilities  of  each  crewmember 
are  summarized  below  (JUAVTOPS,  1997): 

1.  Mission  Commander  (MC) 

The  MC  is  typically  a  rated  Naval  Aviator  or  Naval  Flight  Officer,  who  has  the 
supervisory  responsibility  for  the  UAV  mission.  This  includes  organizing  the  entire  flight 
crew,  coordination  with  external  agencies  and  supported  units  during  pre-mission 
planning,  execution,  and  post  mission  debriefing. 

2.  Internal  Pilot  (IP) 

Typically  a  senior  enlisted  aviation  rating,  who  flies  the  UAV  down  range, 
monitors  instruments  to  ensure  proper  operation,  and  assists  the  payload  operator  (PO)  to 
get  optimal  camera  position.  The  IP  is  also  responsible  for  in-flight  emergencies. 

3.  Payload  Operator  (PO) 

The  PO  is  an  enlisted  operator  who  controls  the  UAV  camera  and  monitors  the 
tracker  bay  to  insure  proper  orientation.  The  PO  assists  the  IP  through  visual  navigation 
and  during  in-flight  emergencies. 

4.  External  Pilot  (EP) 

The  EP,  typically  an  enlisted  operator,  flies  the  UAV  during  launch  and  recovery 
operations.  He  coordinates  the  UAV  handoff  to  the  IP,  and  handles  all  launch  and 
recovery  emergencies. 

5.  Other  Crewmembers 

Depending  on  the  complexity  of  the  mission,  and  the  experience  of  the  crew, 
additional  personnel  may  be  required  to  augment  the  basic  crew.  Intelligence  personnel 

may  be  involved  to  exploit  the  video  imagery  and  pass  that  information  on  to  the 
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appropriate  supported  units.  If  the  mission  calls  for  fire  support  adjustment,  an  artillery 
or  naval  gunfire  forward  observer  will  be  added  to  the  crew.  The  crew  is  then  rounded 
out  with  UAV  maintenance  and  communications  personnel. 
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APPENDIX  B:  MISHAP  SIMULATION  CODE 

MishapSim( ) 

function (runs,  lamda,  PHat,  fltHours) 

{ 

##### 

#  Input: 

#  runs  is  the  number  of  repetitions  for  the  simulation 

#  lamda  is  the  inter-arrival  mishap  rate  for  an  exponential 

#  distribution. 

#  PHat  is  a  vector  of  probabilities  for  a  class  A,  B  or  C 

#  mishap  as  calculated  by  the  ClassSim(  )  function. 

#  fltHours  is  the  length  of  the  flight  period  to  be 

#  simulated. 

# 

#  Function: 

#  This  simulation  is  based  on  an  annual  flight  hour  period. 

#  A  random  exponential  variable  with  the  inputted  rate 

#  parameter  is  used  to  simulate  the  occurrence  of  a  mishap. 

#  This  is  used  to  increment  the  time . 

# 

#  Once  a  mishap  is  generated,  a  second  random  sample  is 

#  drawn  to  determine  the  type  of  mishap  (class  A,  B  or  C)  . 

#  The  damage  cost  is  also  determined  by  the  cost 

#  distribution  data  that  is  hardwired  into  the  program. 

# 

#  A  cost  vector  and  readiness  index  vector  are  built  as 

#  each  annual  flight  period  is  completed.  The  weighted 

#  readiness  index  weighs  a  class  A  =  3,  class  B  =  2,  class 

#  C  =1.  The  maximum  index  in  order  to  be  "MISSION  READY" 

#  is  21.  A  readiness  index  greater  than  21  indicates  "NOT 

#  MISSION  READY" . 

# 
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#  Returns: 

#  The  Function  returns  a  vector  of  simulated  annual  costs, 

#  the  mean  and  standard  deviation  of  the  mishap  cost  and 

#  the  average  readiness  index  for  the  year. 

##### 

readiness  <-  vector (" integer " ,  runs) 
totalCost  <-  vector ( "double" ,  runs) 
for(i  in  l:runs)  { 
time  <-  0 

mi shapC lass  <-  c("A\  "B",  "C" ) 

cost  <-  0 

numA  <-  0 

numB  <-  0 

numC  <-  0 

while (time  <  fltHours)  { 

time  <-  time  +  rexp(l,  lamda/1000) 
mishapType  <-  sample (mishapClass,  1,  replace 
=  T,  PHat) 

if (mishapType  ==  "A")  { 

damage  <-  rnorm(l,  812000,  187000) 
numA  <-  numA  +  1 

} 

if (mishapType  ==  "B" )  { 

damage  <-  max(200000,  rnormfl,  480000, 
214000)) 

numB  <-  numB  +  1 

} 

if (mishapType  ==  "G")  { 

damage  <-  max (10000,  rnorm(l,  88000, 
64000) ) 

numC  <-  numC  +  1 

> 

cost  <-  cost  +  damage 

} 

totalCost[i]  <-  cost 
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readiness [i]  <-  1 

readyFactor  <-  3  *  numA  +  2  *  numB  +  numC 
if (readyFactor  >  21)  { 
readiness [i]  <-  0 

} 

} 

mishapCost  <-  mean ( totalCost ) 

SDCost  <-  sqrt (var (totalCost) ) 
readylndex  <-  mean (readiness) 

return (totalCost,  mishapCost,  SDCost,  readylndex) 

} 
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